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Executive  Summary 

Chapter  One 
Executive  Summary 

1.0  SUMMARY 

This  report  details  the  second  phase  of  the  Office  of  Aviation  Medicine  (AAM)  research  program  on 
Human  Factors  in  Aviation  Maintenance.  This  on-going  research  program  enhances  human  performance 
in  the  aviation  maintenance  system  to  ensure  continuing  flight  safety  and  operational  efficiency.  The 
research  program,  as  reported  in  the  Phase  I  report  (Shepherd,  et  al.,  1991)  was  initially  planned  to  have 
4  steps,  with  feedback  mechanisms  as  shown  in  Figure  1.1.  Phase  I  focused  on  preliminary  investigation 
and  problem  definition  of  human  performance  in  airline  maintenance  environments.  This  Phase  II  report 
describes  research  that  centers  on  the  development  of  hardware  and  software  prototypes  with  potential  to 
enhance  human  performance  in  aircraft  maintenance.  (In  this  report  the  activities  subsumed  under 
"inspection"  are  considered  to  be  part  of  "aircraft  maintenance"). 

The  research  reported  here  has  been 
conducted  by  a  multi-disciplinary  team 
of  scientists  and  engineers  from 
industry  and  academia.  The  research 
team  has  worked  in  very  close 
cooperation  with  the  international 
aviation  industry,  mostly  with  US 
airlines  and  aviation  manufacturers. 

This  report  includes  seven  chapters  and 
can  be  considered  as  an  edited  volume 
in  that  each  chapter  is  written  to  stand 
alone  as  the  work  of  each  research 
group. 


1.1  CONTINUING  RATIONALE 

Shepherd  et  al.  (1991)  and  Shepherd  &  Johnson  (1991),  offered  an  extensive  description  of  the  rationale 
for  the  research  program.  These  reports  described  the  complexity  of  the  total  aviation  maintenance  system 
and  the  role  of  the  human  within  the  system.  Increased  maintenance  workload,  caused  in  part  by  an 
increased  level  of  air  carrier  operations,  is  one  reason  to  focus  on  improving  aviation  maintenance 
technician  (AMT)  performance.  The  challenge  of  providing  continuing  air  worthiness  of  the  aging  fleet 
while  developing  knowledge  and  skills  for  maintaining  new  technology  aircraft  places  a  burden  on  airline 
maintenance  organizations. 

Phase  1  research  investigated  methods  for  enliancing  human  performance  in  aircraft  maintenance.  There 
are  ample  reasons  for  continuing  these  investigations.  For  example,  operations  will  continue  to  increase. 
Airlines  will  fly  more  hours  with  the  same  fleet  sizes.  Thus,  there  will  be  less  time  for  maintenance  and 


Figure  1.1  The  Research  FYogram 
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greater  stress  on  the  fleet.  Therefore,  enhancing  human  performance  in  maintenance  continues  to  be  an 
important  priority. 

Resources  are  finite.  Airlines,  during  1991-92,  have  not  been  profitable.  Since  the  Phase  I  report  was 
published,  major  air  carriers  such  as  Pan  Am  and  regional  carriers  such  as  Midway  Airlines  have  gone 
out  of  business.  Other  carriers  have  suffered  record  financial  losses  and  face  uncertain  futures.  Airlines 
recognize  the  criticality  of  cost  control  in  every  aspect  of  their  operation.  However,  cost  control  cannot 
Jeopardize  safety.  This  research  program  recognizes  that  the  enhancement  of  human  performance  in 
maintenance  is  critical  to  the  safety  and  efficiency  of  air  carrier  operations. 

1.1.1  Integration  of  Human  Factors  Research  Efforts 

Research  to  enhance  human  performance  in  aircraft  maintenance  can  focus  on  several  dimensions,  such 
as  the  human,  the  tools,  the  work  place,  work  procedures,  and  management  philosophies.  The  research 
must  be  useful  to  maintenance  practitioners  as  well  as  to  the  human  factors  research  community.  This 
report,  therefore,  has  practical  as  well  as  scientific  value. 

1.2  ADVANCED  TECHNOLOGY  TRAINING  (Chapter  Two) 

Advanced  technology  gaining  combines  artificial  intelligence  technology  with  conventional  computer- 
based  training.  The  technology  was  described  extensively  in  the  Phase  1  report  (Shepherd,  et  al.,  1991) 
and  elsewhere  (Johnson  &  Norton,  1991  and  Johnson  &  Norton,  in  press). 

This  chapter  describes  the  continuing  effort 
that  has  converted  a  Phase  I  training 
prototype  to  a  fully  operational  advanced 
technology  training  system  for  the  Boeing 
767-3(X)  environmental  control  system 
(ECS).  The  system  is  simulation-based  in 
that  it  permits  the  user  to  access  and 
operate  all  panels,  controls,  and  built-in¬ 
test  equipment  of  the  ECS.  Figure  1.2 
shows  the  human-computer  interface  for 
the  ECS. 

The  ECS  trainer  is  unique  not  only 
becau.se  of  the  simulation  but  al.so  because 
of  the  robust  software  used  for  modeling 
student  performance  and  providing 
feedback,  explanation,  and  remediation. 

These  modeling  features  are  described  in 
Chapter  Two. 

In  addition  to  providing  simulation,  the  ECS  Tutor  provides  on-line  acce.ss  to  the  training  manual  for  the 
ECS.  The  software  makes  it  easy  for  the  student  to  use  the  manual  during  training.  This  research  is 
preparing  for  development  of  an  integrated  information  system  which  can  provide  not  only  training  but 
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also  real-time  job  aiding  and  maintenance  documentation.  Research  related  to  the  concept  of  on-line 
documentation  is  also  described  in  Chapter  Four. 

The  chapter  also  describes  the  process  of  formative  evaluation  that  took  place  as  the  training  systems 
underwent  many  iterations  with  software  engineers  and  training  professionals  at  Delta  Air  Lines  and  at 
Clayton  State  College.  A  substantive  training  effectiveness  evaluation  will  be  conducted  at  Delta. 

Finally,  as  advanced  technology  training  systems  become  more  commonplace,  it  is  likely  that  they  can 
be  used  for  AMT  certification.  Therefore,  this  chapter  also  reports  on  the  research  implications  of  the 
pending  changes  to  Federal  Aviation  Regulation  (FAR)  Parts  65  and  147. 

1.3  ADVANCED  TECHNOLOGY  MAINTENANCE  JOB  AIDS  (Chapter  Three) 

This  chapter  addresses  existing  approaches  to  job  aiding  in  maintenance,  the  drawbacks  to  such 
approaches,  the  prospects  for  using  emerging  technologies  to  develop  maintenance  job  aiding  systems,  and 
the  impact  of  emerging  technologies  on  human  performance.  There  were  two  major  themes  to  the 
research:  1)  many  previous  attempts  at  building  maintenance  job  aids  consisted  of  trying  to  replace 
human  expertise  with  machine  expertise;  and.  2)  problems  with  such  approaches  have  led  to  a 
reconsideration  of  the  skills  and  abilities  of  human  operators  and  ways  to  capitalize  on  them. 

Accordingly,  the  chapter  calls  for  a  ‘cooperative  system’  approach  to  designing  such  systems;  a 
cooperative  system  is  one  in  which  a  human  and  a  computer  are  actively  involved  in  the  problem  solving 
process.  The  chapter  presents  a  study  which  used  this  approach  in  developing  a  job  aid.  Some  of  the 
results  of  the  study  that  are  relevant  to  designing  maintenance  job  aids  and  integrated  information  systems 
(Johnson  &  Norton.  1992  a  &  b)  are  also  presented.  Finally,  a  research  and  development  plan  for  building 
a  maintenance  job  aid  for  aircraft  maintenance  is  discussed. 

1.3.1  Human  Performance  Implications  of  Artificial  Intelligence  Approaches 

The  bulk  of  the  job  aiding  systems  encountered  in  a  literature  review  used  artificial  intelligence  and  expert 
systems  techniques.  While  artificial  intelligence  techniques  can  provide  a  computer  with  powerful  problem 
solving  abilities,  job  aiding  systems  which  rely  solely  on  such  techniques  often  meet  with  limited 
operational  success.  One  of  the  reasons  for  such  limited  success  is  that  the  computer  is  supposed  to 
embody  the  knowledge  and  abilities  of  a  human  expert,  when,  in  fact,  such  systems  are  necessarily 
incomplete.  Because  builders  of  expert  systems  cannot  capture  all  of  the  human  expert’s  knowledge  about 
a  task,  such  systems  often  draw  erroneous  conclusions.  Therefore,  the  operator  must  have  enough 
expertise  to  realize  that  the  computer  is  wrong;  the  problem  is  that  the  operator  will  not  develop  such 
expertise  unless  he/she  is  actively  involved  in  the  problem  solving  process.  However,  these  problems  do 
not  exclude  artificial  intelligence  techniques  from  use  in  operational  job  aids.  Rather,  tlie  question  is  one 
of  emphasis:  instead  of  using  artificial  intelligence  techniques  as  the  foundation  for  a  job  aid,  they  should 
be  used  in  conjunction  with  other  methods  of  performance  aiding  (e.g..  representation  aiding). 

1.3.2  Human  Performance  with  a  Cooperative  System 

A  re.search  study  (Layton.  1992)  which  investigated  human  performance  with  three  forms  of  a  cooperative 
system  provided  some  interesting  insights  into  how  such  systems  affect  human  behas  ior.  Tliis  system  was 
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designed  to  assist  commercial  airline 
pilots  and  dispatchers  in  enroute  flight 
planning.  (Figure  1.3  depicts  a 
portion  of  the  system  displays  and 
controls.)  This  research  has  provided 
some  interesting  insights  into  the  ways 
in  which  job  aiding  tools  affect  human 
performance. 


1.3J  Research  and  Development 
Plan 


A  three-phase  plan  for  developing  an 
aviation  safety  inspector  job  aid  using 
cooperative  system  techniques  was 
developed.  Initial  interactions  with  the 
Flight  Standards  Service  suggest  that 
the  job  aid  will  assist  inspectors  in 
researching  operator  information  and 
documenting  inspection  activities.  The  system  will  link  inspection  forms  so  that  information  that  is 
entered  into  one  form  will  automatically  be  entered  into  the  other  forms  being  used.  The  system  will  also 
provide  the  ability  to  search  for  information  using  on-line  documentation.  Such  documentation  may 
include  the  federal  aviation  regulations,  advisory  circulars,  airworthiness  directives,  FAA  policies,  and 
operator-specific  information.  The  system  will  have  a  cellular  modem  capability  so  that  it  can  connect 
to  the  Flight  Standards  mainframe  system  and  will  likely  use  a  CD  ROM  to  store  much  of  the  on-line 
documentation. 

1.4  DIGITAL  DOCUMENTATION  (Chapter  Four) 

Maintenance  personnel  are  often  overwhelmed  with  the  amount  of  technical  documentation  necessary  to 
accomplish  a  given  task.  The  information  comes  from  a  variety  of  sources  including  company  and 
manufacturer’s  manuals,  and  government  documents,  like  advisory  circulars  or  regulations.  Currently  most 
maintenance  documentation  exists  as  hard  copy  or  microfiche.  The  task  of  keeping  these  databases  current 
is  very  time  consuming  and  expensive. 

This  research  task,  called  the  Hypermedia  Information  System.  (HIS)  shown  in  Figure  1.4,  studies 
advanced  technology  software  and  hardware  techniques  for  information  storage  and  retrieval.  The  primary 
products  of  this  research  will  be  techniques  for  the  development  and  use  of  large  information  sources  on 
small  portable  computer  systems. 

The  term  "hypermedia"  refers  to  a  combination  of  text,  graphics,  animation,  audio,  and  video  to  convey 
information.  Such  information  bases  are  designed  to  be  accessed  easily,  usually  in  a  non-linear  fashion. 
This  hypermedia  research  will  make  it  possible  for  a  technician  to  access  a  manual  for  all  media  and 
information  to  complete  a  job.  The  research  fosters  co-development  of  integrated  information  systems 
(Johnson  &  Norton  1992,  a  &  b)  that  provide  training,  job-aiding,  and  on-line  documentation. 
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Figure  1.3  Enroute  flight  planning  cooperative  system 
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A  by-product  of  the  research  is  the 
development  of  a  hypermedia 
information  system  for  all  technical 
publications  from  the  Aviation 
Medicine  Human  Factors  in  Aviation 
Maintenance  research  program. 

Ultimately  this  digital  source  of 
information  wilt  be  published  on  a  CD- 
ROM  (Compact  Disc-Read  Only 
Memory). 

The  proceedings  of  one  of  the  first  six 
conferences  on  aviation  maintenance 
human  factors  has  already  been 
prepared  for  distribution  as  a  digital 
document.  The  seventh  conference 
will  be  the  first  time  that  the  meeting 
proceedings  will  be  distributed  in  Figure  1.4  Hypermedia  Information  System 
digital  format  at  the  meeting.  The 

software  developed  through  this  research  effort  has  facilitated  the  timely  publication  of  such  digital 
documentation. 

1.5  HUMAN  RELIABILITY  IN  AIRCRAFT  INSPECTION  (Chapter  Five) 

The  research  related  to  improving  human  reliability  in  aircraft  inspection  built  upon  the  solid  task  analytic 
foundation  derived  under  Phase  I.  The  chapter  describes  two  studies;  one  study  related  to  the  re-design 
of  workcards  for  inspection  and  the  other  a  study  of  the  lighting  environment  for  inspection.  Both  studies 
offer  practical  human  factors  guidelines  applicable  to  these  topics.  The  chapter  also  describes  a  plan  to 
consider  human-computer  interface  issues  applicable  to  computer-based  maintenance  aids. 

This  chapter  also  describes  a  series  of  laboratory  experiments  that  evaluate  the  effects  of  time  pressure 
on  inspection  and  the  improvement  of  training  techniques  for  visual  inspection.  The  chapter  describes  a 
study  of  the  classification  of  human  error  in  inspection.  The  classification  is  particularly  valuable  in  its 
review  of  many  scientific  studies  of  human  error.  These  studies  form  the  basis  for  the  team’s  development 
and  presentation  of  system  models  of  human  enor  in  inspection.  These  models  provide  the  means  to 
understand,  predict,  report  and  manage  inspection  errors. 

Inspection  is  information  processing.  The  chapter  reports  research  on  the  design  of  information  flow  in 
the  inspection  environment.  The  research  helps  to  determine  what,  when,  and  how  to  present  information 
to  the  inspector.  Experimental  results  are  presented  regarding  optimal  methods  of  information  presentation 
in  inspection  tasks. 

Chapter  Five  also  de.scribes  a  joint  study  of  inspection  practices  in  the  UK  and  USA.  The  comparative 
study  observed  that  management  structures  of  maintenance  and  inspection  are  more  closely  intermeshed 
in  the  UK  than  in  the  US.  Other  differences  and  rationales  are  reported. 
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1.6  GUIDELINES  FOR  HUMAN  FACTORS  IN  MAINTENANCE  (Chapter  Six) 

Human  Factors  principles  are  often  derived  in  laboratory  studies  of  procedures,  equipment,  effects  of  time, 
temperature,  lighting  and  other  variables.  Much  of  the  information  derived  from  these  studies  is  reported 
for  scientists,  psychologists,  and  engineers  for  academic  applications.  This  task  is  reviewing  the  human 
factors  literature  from  a  wide  variety  of  parallel  and  similar  areas  to  aircraft  maintenance.  All  of  the 
research  results  from  the  Aviation  Medicine  Aircraft  Maintenance  Human  Factors  program  will  be 
combined  with  this  information  base  to  produce  a  Human  Factors  Guide  for  Aircraft  Maintenance.  This 
guide  promises  to  be  useful  to  airline  maintenance  management  system  designers.  FAA  oversight 
personnel,  and  others  as  they  strive  to  improve  human  performance  in  the  maintenance  system.  Chapter 
Six  offers  an  example  chapter  from  the  Human  Factors  Guide. 

1.7  CREW  RESOURCE  MANAGEMENT  FOR  MAINTENANCE:  EVALUATION  OF  A 
TRAINING  PROGRAM  (Chapter  Seven) 

Phase  I  (Shepherd,  et  al..  1991)  reported  on  management-worker  communications  in  the  aviation 
maintenance  environment.  Phase  II  research  has  shifted  focus  to  the  effects  of  crew  resource  management 
(CRM)  training  in  an  airline  maintenance  environment.  The  research  has  concentrated  on  communication 
among  maintenance  crews.  The  researcher  participated  in  the  evaluation  of  the  effectiveness  of  a 
particular  airline’s  CRM  training  for  maintenance  personnel  and  in  the  post-training  performance  effects 
on  maintenance  managers  and  technicians. 

The  CRM  course  acceptance  has  been  very  high.  In  fact  maintenance  crews  have  demonstrated  greater 
acceptance  of  the  CRM  principles  than  have  flight  crews.  The  research  indicates  that  relevant  attitudes 
about  CRM  improved  immediately  after  training.  Course  attendees  have  repc)ried  that  the  CRM  principles 
have  caused  them  to  be  more  actively  involved  in  all  maintenance  decision  making. 

The  CRM  evaluation  research  is  valuable  in  that  it  has  created  insu'uments  and  criteria  to  measure  post- 
training  maintenance  performance.  These  measures  will  be  helpful  to  assess  the  U’aining  and  cost 
effectiveness  of  such  human  performance  enhancement  courses. 

1.8  CONTINUED  COMMUNICATION 

The  seven  workshops  that  have  been  conducted  to  date  under  the  Aviation  Medicine  research  program 
have  facilitated  communication  between  researchers  and  industry.  The  immediate  application  of  some  of 
the  research  activities  described  above  will  allow  the  industry  to  increase  reliability  and  lower  costs  The 
Office  of  Aviation  Medicine  (AAM)  intends  to  continue  sponsorship  of  itie  workshops  throughout  the 
duration  of  the  research  program. 

The  participation  and  cooperation  of  the  airline  industry  has  been  instrumental  to  tlte  AAM  research 
program.  Air  carriers,  manufacturers,  and  schools  have  been  extremely  cooperative  and  helpful.  Tliis 
cooperation  is  gratefully  acknowledged. 
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Chapter  Two 

Advanced  Technology  Training  for  Aviation  Maintenance 
2.0  INTRODUCTION 

As  technology  advances,  the  job  of  the  aircraft  maintenance  technician  (AMT)  becomes  increasingly 
difficult.  The  AMT  must  deal  with  new  technology  (digital  components,  composite  materials,  etc.)  as  well 
as  an  aging  aircraft  fleet.  Concurrently,  the  AMT  work  force  dwindles  (Shepherd,  et  al.,  1991). 
Therefore,  maintenance  training  must  respond  to  these  challenges  by  inaeasing  the  effectiveness  of  the 
current  work  force.  This  chapter  outlines  how  the  Office  of  Aviation  Medicine  (AAM)  uses  advanced 
technology  training  to  address  these  issues. 

2.1  RESEARCH  PHASES 

The  advanced  technology  training  research  began  in  January  of  1990.  This  earlier  phase  of  the  work 
assessed  the  status  of  training  technology  for  maintenance  technicians.  Based  upon  this  assessment,  the 
AAM  built  a  prototype  intelligent  tutoring  system  (ITS)  for  the  Environmental  Control  System  (ECS) 
(Shepherd,  et  al.,  1991).  See  (Poison  and  Richardson,  1988)  for  more  information  on  ITS. 

The  current  phase  of  the  research  expands  this  prototype  to  be  an  operational  tutoring  system  for  the  ECS. 
In  order  to  provide  a  measure  of  the  effectiveness  of  the  Tutor,  this  phase  also  designs  an  evaluation  of 
the  tutor.  Finally,  as  advanced  technology  training  systems  become  more  commonplace,  it  is  likely  that 
they  can  be  used  for  AMT  certification.  Therefore,  this  phase  also  reports  on  the  research  implications 
of  the  mechanic  certification  rules  changes  to  Federal  Aviation  Regulation  (FAR)  Parts  65  and  147. 

It  is  important  that  the  Tutor  be  an  effective  training  tool.  Despite  formative  evaluation  of  the  Tutor 
throughout  the  second  phase,  a  formal  evaluation  will  be  conducted  during  the  third  phase.  This  will 
evaluate  the  Tutor  for  user  acceptance,  training  effectiveness,  and  cost  effectiveness.  In  addition,  the  third 
phase  of  the  project  will  investigate  the  use  of  advanced  technology  training  for  psychomotor  activities. 

2.2  ADVANCED  TECHNOLOGY  TRAINING  TUTOR 

The  Advanced  Technology  Training  research  developed  an  operational  tutoring  system  for  the  ECS  of  the 
Boeing  767-300,  as  shown  in  Figure  2.1.  This  section  describes  the  features  of  the  Tutor,  the  design  of 
the  Tutor,  and  the  lessons  learned  while  developing  the  operational  Tutor. 

2.2.1  Description  of  the  ECS  Tutor 

In  the  ECS  Tutor,  the  student  interacts  with  panels,  conftols,  test  equipment,  manuals,  and  displays.  These 
graphics  are  meant  to  simulate  the  "look  and  feel"  of  the  real  ECS  components.  A  simulation  of  the  ECS 
responds  to  the  student’s  actions  by  updating  the  appropriate  data  values  on  the  Engine  Indicating  Crew 
Alerting  System  (EICAS)  display  and  the  Overhead  Panel,  as  shown  in  Figure  2.2  and  Figure  2.3, 
respectively. 

The  system  operates  in  two  distinct  modes:  Normal  Operation  and  Malfunction.  During  normal  operation, 
the  ECS  simulation  lets  the  student  see  the  proper  operation  of  the  system.  During  a  malfunction,  the  ECS 
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will  exhibit  the  symptoms  associated  with  the  current 
malfunction.  The  student  controls  whether  the 
system  operates  in  normal  mode  or  malfunction 
mode. 

Regardless  of  the  mode,  the  student’s  interaction 
with  the  system  is  similar.  In  both  modes,  the 
student  has  access  to  all  of  the  tools  needed  to 
operate  the  ECS.  The  student  interacts  with  control 
panels,  information  displays,  built-in  test  equipment, 
component  information,  etc.  However,  malfunction 
mode  is  unique  in  several  respects. 


Figure  2.1  ECS  Overview  Screen 


Figure  2.2  EICAS  Display  Figure  2.3  Overhead  Panel 

As  a  malfunction  begins,  the  student  sees  a  description  of  the  problem,  with  the  accompanying  fault  code. 
After  seeing  the  description,  the  student  may  order  replacement  parts  to  be  delivered  to  the  plane.  If  the 
students  does  not  order  a  part  that  they  need,  when  they  go  to  replace  the  pan  they  must  wait  for  the  pan 
to  arrive.  The  Tutor  simulates  this  wait  by  removing  15  minutes  from  the  time  that  remains. 

During  a  malfunction,  the  student  has  access  to  the  Fault  Isolation  Manual  (FIM),  shown  in  Figure  2.4. 
The  FIM  is  the  fault  tree  that  the  AMT  follows  while  troubleshooting.  It  outlines  the  tests  and  procedures 
that  should  be  performed  while  diagnosing  a  failure.  While  troubleshooting  from  the  FIM,  the  student 
must  perform  the  tests  in  the  order  prescribed  by  the  FIM. 

Even  though  the  FIM  is  the  recommended  method  of  troubleshooting,  the  EC.S  Tutor  supports  a  more 
flexible  way  to  troubleshoot.  The  Tutor  provides  a  schematic  of  the  cooling  pack  from  which  the  student 
may  troubleshoot.  The  student  selects  a  component  (in  any  order)  from  the  schematic  and  then  choo.ses 
whether  to  Inspect.  Test,  or  Replace  the  component. 


10 


Advanced  Technology  Training  for  Aviation  Maintenance 

In  both  the  FIM  and  the  cooling  pack 
schematic,  the  Tutor  records  the 
student’s  actions.  At  any  time,  the 
student  may  ask  for  advice  on  how  to 
troubleshoot  the  current  malfunction. 

The  tutor  compares  the  student’s 
actions  with  the  actions  of  an  expert. 

The  Tutor  suggests  the  appropriate 
next  step  to  the  student. 

When  the  student  replaces  a  faulty 
component,  the  Tutor  updates  the 
simulation  to  reflect  proper  operation. 

However,  just  as  on  the  flight  line,  the 
student  must  verify  that  the 
replacement  was  effective.  When  the 
students  are  confident  of  success,  they 
can  have  the  Tutor  check  their  Figure  2.4  Fault  Isolation  Manual  Display 
solution. 

2.2.2  Design  of  the  ECS  Tutor 

During  the  design  of  the  ECS  Tutor,  several  separate  design  issues  were  addressed.  These  issues  include 
interface  design,  instructional  design,  and  simulation  design.  This  section  also  describes  the  tools  used 
to  implement  this  design. 

2.2.2.1  Interface  Design 

The  design  of  the  ECS  Tutor  ensures  that  the  majority  of  relevant  troubleshooting  information  is  only  one 
mouse  click  away  from  the  primary  display,  shown  in  Figure  2.5.  From  this  Overview  display  students 
may  access  the  displays,  controls,  and  components  that  they  need  to  troubleshoot  a  malfunction. 

Every  troubleshooting  screen  in  the  Tutor  adheres  to  a  standard  format,  which  consists  of  a  grey  border 
on  the  edge  of  the  screen.  The  border  contains  functions  that  are  useful  for  all  troubleshooting  screens. 
The  left-hand  border  consists  of  three  different  areas  -  the  Status  area,  the  Navigation  (or  "Go  To")  area, 
and  the  Help  Area.  The  bottom  border  contains  a  Message  Area. 

The  top-most  portion  of  the  border  is  the  Status  area.  This  area  notifies  the  student  of  the  current 
operating  mode  -  "Normal"  or  "Malfunction".  During  a  Malfunction,  this  area  also  indicates  the  amount 
of  time  that  remains  for  troubleshooting.  The  center  portion  of  the  border  is  the  Navigation  area.  These 
three  buttons  allow  the  student  to  immediately  go  to  one  of  three  commonly  used  screens  -  the  Overview 
Screen,  the  Fault  Isolation  Manual,  or  the  Aircraft  Log  Sheet. 

The  lower  portion  of  the  border  is  the  Help  area.  This  area  assists  the  student  in  four  different  ways: 
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•  How  to  use  the  current  screen 

•  Information  about  mechanical,  electrical,  pneumatic  concepts 

•  How  to  navigate  through  the  system 

•  What  to  do  next  -  advice  during  troubleshooting 

Finally,  the  border  that  runs  along  the 
bottom  of  the  screen  is  the  Message  area. 

As  the  student  moves  the  cursor  across 
selectable  regions,  the  Message  area  shows 
additional  information  about  that  selectable 
region.  For  example,  the  Overview  display 
contains  a  button  labeled  "Check 
Solution".  When  the  cursor  enters  this 
button,  the  Message  area  will  read  "Info: 

Click  this  button  when  you  think  you  have 
solved  the  problem." 

2.2 J,2  Instructional  Design 

The  instructional  design  of  the  ECS  Tutor 
was  improved  in  two  ways.  First,  a  lesson 
orientation  provides  more  context  for  each 
of  the  malfunctions.  Also,  the  "over-the- 
shoulder"  advice  helps  students  who  are  not  making  progress  toward  a  solution.  The  following  sections 
describe  each  in  more  detail. 

2.2.2.2.1  Lesson  Orientation 

The  instructional  design  for  the  ECS  Tutor  centers  around  the  idea  of  a  lesson,  where  a  lesson  is  a  logical 
collection  of  malfunctions.  Since  the  Fault  Isolation  Manual  (FIM)  is  driven  by  symptoms,  the  ECS  Tutor 
assembles  lessons  according  to  the  symptoms  for  each  malfunction  (e.g.  PACK  OFF/INOP  lights) .  As 
the  Tutor  presents  each  lesson,  it  provides  the  student  with  information  similar  to  that  shown  in  Figure 
2.6. 

The  Background  Information  section  describes  the  bases  for  the  indicator  lights.  This  section  gives  the 
common  causes  for  the  symptom.  The  Systems  section  gives  a  list  of  systems  that  relate  to  the  indicator 
light.  The  user  may  choose  one  of  these  items  to  get  more  detailed  information  about  a  specific  system. 
The  Objects  section  provides  a  list  of  components  that  may  be  useful  in  solving  the  current  malfunction. 
The  user  may  choose  one  of  these  items  to  get  more  information  about  that  component. 

After  completing  a  lesson,  the  Tutor  reviews  the  student’s  performance  from  the  previous  lesson.  If  the 
Tutor  detects  any  deficiencies,  it  will  provide  material  that  will  help  the  student  overcome  the  deficiencies. 


Figure  2.5  ECS  Overview  Display 
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2.22.2.2  Over-the-Shoulder 
Advice 

Another  addition  to  the  ECS  Tutor 
makes  the  advice  that  the  student 
receives  "smarter".  In  the  Prototype, 
the  student  received  advice  only  on 
demand.  In  the  Phase  Two  Tutor,  the 
system  monitors  the  student’s  progress 
toward  a  solution  and  offers  help  to  the 
student  if  the  student  does  not  appear 
to  be  making  satisfactory  progress. 

The  student  may  choose  to  see  this 
"over-the-shoulder"  advice  or  ignore  it. 

2.2.23  Simulation  Design 

The  simulation  provides  realistic  Figure  2.6  Lesson  Overview  Screen 
responses  to  a  student’s  actions.  For 

example,  the  ECS  adjusts  output  temperature  of  the  cooling  pack  when  the  student  alters  the  desired 
zone’s  temperature.  The  new  data  values  appear  on  the  Overhead  Panel  and  the  EICAS  display.  The  way 
in  which  the  simulation  accomplishes  this  temperature  change  should  be  transparent  to  the  student. 

In  Phase  One,  the  prototype  "simulation"  consisted  of  a  table  of  data  values  for  certain  predefined 
conditions.  The  data  shown  to  the  student  was  updated  according  to  this  table.  This  approach  was 
adequate  for  the  prototype  because  of  the  limited  scope  of  the  prototype.  However,  due  to  the  expanded 
number  of  potential  malfunctions,  the  Phase  Two  operational  tutor  needed  a  more  robust  simulation  of  the 
ECS  environment. 

Phase  Two’s  simulation  contains  a  model  of  individual  components.  The  simulation  acts  upon  each 
component’s  inputs  to  produce  that  component’s  outputs.  This  approach  allows  a  malfunction  to 
propagate  through  the  system.  For  example,  if  a  valve  fails,  it  will  affect  its  outputs  (such  as  air  flow). 
The  component  that  is  downstream  from  the  valve  will  receive  unsatisfactory  air  flow  and  produce  an 
erroneous  output.  The  next  component  will  receive  this  as  its  input,  and  the  fault  will  continue  to 
propagate. 

The  simulation  will  act  upon  inputs  to  produce  outputs,  whether  those  inputs  are  good  or  bad.  This  design 
allows  for  the  same  simulation  to  be  used  in  both  normal  operation  as  well  as  during  malfunctions. 

2.2.2.4  Development  Tools 

The  ECS  Tutor  uses  a  variety  of  software  tools.  The  Tutor  uses  different  tools  to  create  the  interface,  the 
simulation,  and  the  graphics,  as  described  below. 

The  interface  was  developed  using  Asymetrix  Toolbook.  Toolhook  supports  quick  and  easy  interface 
development  in  the  Microsoft  Windows  environment. 
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The  Prototype  simulation  also  used  Toolhook.  However,  as  the  simulation  matured,  it  migrated  from 
Toolhook  to  the  "C"  programming  language.  "C"  provided  greater  speed  and  more  llexibility  than 
Toolhook. 

The  graphics  used  in  the  operational  tutor  combine  many  different  graphics  creation  techniques.  Some 
graphics  were  custom-designed  using  graphics  packages,  while  others  were  scanned  from  existing  training 
documentation.  The  Tutor  also  used  photographs  taken  with  a  digital  camera  and  stored  as  graphics  files. 
As  the  training  display  hardware  advances,  digital  photographs  will  become  cri.sper  and  more  realistic. 

2.2  J  Lessons  Learned  from  the  Development  of  the  Tutor 

As  the  Tutor  progressed  from  the  Prototype  to  the  complete,  operational  system,  formative  evaluations 
highlighted  several  different  areas  that  needed  to  be  addressed;  advice,  instructional  motivation,  and 
student  confidence  with  the  interface.  A  de.scription  of  each  area  follows. 

2.2J.1  Advice 

When  the  ftototype  gave  advice  to  the  student,  the  advice  told  the  student  what  procedure  to  perform 
next.  It  followed  the  FIM  exactly.  However,  it  did  not  provide  enough  motivation  as  to  why  ti>  perform 
a  certain  procedure.  The  current  tutor  enhances  the  advice  to  help  answer  the  question  "Why  perform  this 
procedure  instead  of  a  different  procedure  ?"  With  this  extra  information,  the  student  learns  the  logic 
behind  the  FIM,  instead  of  blindly  following  procedures. 

2.232  Instructional  Motivation 

The  Prototype  emphasized  troubleshooting,  but  lacked  instructional  focus.  During  this  phase,  the  Tutor 
adds  more  emphasis  to  instructional  issues.  As  described  above  in  Section  2.2.2,2,  each  lesson  contains 
background  information,  system  information,  component  information.  Much  of  this  information  was 
available  for  the  Prototype,  but  the  student  had  to  search  for  it.  The  Tutor  now  presents  the  information 
to  the  student  in  a  more  directed  manner.  This  method  reinforces  to  the  student  which  pieces  of 
information  are  important  under  dilferent  circum.stances. 

2.2JJ  Student  Confidence  with  the  Interface 

As  students  used  the  system,  especially  for  the  first  time,  they  were  hesitant  to  click  t)n  buttons  with  the 
mouse  becau.se  they  weren’t  sure  what  would  happen.  To  allay  some  their  indecision,  the  Tutor  now 
contains  the  Message  area.  Anytime  the  cursor  enters  a  selectable  region,  a  short  descriptive  message 
appears  in  the  Message  area  that  describes  the  function  of  the  selectable  region.  As  training  developers, 
we  must  be  mindful  to  provide  as  much  rea.s.surance  to  the  student  as  possible. 

2.3  EVALUATION  PLAN 

If  Advanced  Technology  Training  is  to  become  commonplace,  it  must  be  accepted  by  the  user  population. 
ALSO,  if  it  is  acceptable  to  the  AMT’s  it  mu.st  al.so  prove  to  be  an  effective  training  t(X)l.  Tliis  .section 
outlines  a  plan  to  evaluate  the  FXS  Tutor  for  user  acceptance,  cost  effectiveness,  training  effectiveness. 
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2.3.1  User  Acceptance  Testing 

User  acceptance  testing  will  be  conducted  in  the  following  different  user  populations: 

•  Delta  Air  Lines  instructors 

•  Delta  Air  Lines  students 

•  Clayton  State  College  Airframe  &  Powerplant  (A&P)  students 

This  testing  will  use  questionnaires  and  interviews  to  gather  data.  The  ECS  Tutor  will  be  modified  as 
necessary  to  accommodate  problems  areas  that  the  test  identifies. 

2.3.2  Training  Effectiveness  Study 

The  training  effectiveness  study  will  test  the  following  hypr^thesis; 

A  combination  of  conventional,  classroom  leaching  and  the  ECS  tutor  is  more  effective  than 
conventional,  classroom  teaching. 

This  experiment  uses  both  a  control  group  and  an  experimental  group  (from  the  Delta  Air  Lines  student 
population).  Both  groups  will  receive  the  normal  4  to  5  hour  ECS  classroom  training  segment.  The 
control  group  will  receive  an  additional  classroom  session  on  ECS  troubleshooting.  The  experimental 
group  will  receive  the  lesson  on  ECS  troubleshooting  via  the  ECS  Tutor.  After  the  troubleshooting 
session,  each  group  will  take  an  exit  exam  which  will  measure  their  ECS  proficiency.  The  scores  on  the 
exit  exam  will  be  used  as  the  basis  of  comparison.  Figure  2.7  summarizes  the  experiment. 

2.3J  Cost  Effectiveness  Study 

A  cost  effectiveness  study  will  measure 
the  effectiveness  of  the  ECS  Tutor 
along  several  dimensions.  Among 
these  are  both  development  costs  and 
delivery  costs. 

2.4  PART  147  &  PART  65 
RULE  CHANGES  AND 
HUMAN  FACTORS 
IMPLICATIONS 

2.4.1  Part  147  -  Aviation 

Maintenance  Schools 

From  1989  through  1991  there  has  been  a  rule  change  in  the  making  for  FAR  Part  147  -  Aviation 
Maintenance  Technician  Schools.  After  extensive  public  hearings,  the  rule  changes  are  completed  and  will 
be  published  in  1992.  The  changes  in  the  approved  curricula  were  designed  to  be  more  responsive  to  the 
needs  of  today’s  aviation  indu.stry.  For  example,  skill  and  knowledge  requirements  for  gas  turbine  engines 
have  been  increased  while  requirements  related  to  fabric  covered  aircraft  have  been  lessened. 


Figure  2.7  Evaluation  Plan 
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A  second  significant  change  in  FAA  philosophy  regards  educational  media.  The  new  rule  permits  the 
substitution  of  advanced  technology  training  systems  (like  computer-based  training,  CBT)  for  classroom 
and  laboratory  instruction.  This  change  in  the  rule  is  likely  to  foster  an  explosion  of  CBT  for  aviation 
maintenance  training.  The  FAA  should  monitor  the  quality  of  such  CBT  to  ensure  that  content  and  the 
delivery  methods  are  appropriately  applied. 

The  changes  to  Part  147  are  not  significant  in  number.  However,  the  recognition  of  the  potential  for 
computer-based  training  is  a  very  positive  step  in  helping  the  schools  to  train  for  modem  aircraft  systems. 

2.4,2  Part  65  Certification:  Airmen  Other  than  Flight  Crewmembers 

The  public  hearings  and  Aviation  Rulemaking  Advisory  Committees  are  currently  working  with  the  FAA 
to  consider  changes  to  Part  65.  The  changes  that  are  being  considered  address  such  issues  as  experience 
requirements,  training  and  proficiency  requirements,  and  issues  related  to  issuances  of  certification  of 
personnel  for  repair  facilities  outside  of  the  United  States. 

2.4J  Job/Task  Analysis 

All  who  have  been  involved  in  Part  147  and  Part  65  agree  that  there  is  a  dire  need  for  a  new  Job/Task 
Analysis  (JTA)  for  the  Aviation  Maintenance  Occupation.  Such  a  JTA  has  not  been  done  since  1969. 
The  activity  related  to  this  rule  change  is  prompting  the  appropriate  funding  for  the  AMT  JTA. 

2.5  SUMMARY 

This  research  shows  that  Advanced  Technology  Training  can  be  applied  to  the  Aviation  Maintenance 
community.  This  particular  system  concentrated  on  the  ECS  because  of  the  generic  nature  of  the  ECS 
system  (i.e.,  ECS  principles  are  similar  from  aircraft  to  aircraft).  However,  this  same  approach  is 
applicable  in  all  other  aviation  maintenance  areas.  Up  to  this  point,  the  Advanced  Technology  Training 
has  concentrated  on  the  mental  aspects  of  training  (logical  troubleshooting).  As  this  work  continues,  the 
research  will  attempt  to  combine  the  logical  troubleshooting  techniques  with  the  psychomotor  skills 
required  to  maintain  the  aircraft. 
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Chapter  Three 

Emerging  Technologies  for  Maintenance  Job  Aids 
3.0  INTRODUCTION 

Maintenance  is  fast  becoming  one  of  the  most  frequent  application  areas  for  job  aiding.  Maintenance  job 
aids  range  from  automatic  preventive  maintenance  schedulers,  to  systems  that  monitor  equipment  status 
and  recommend  maintenance  based  on  trends  in  equipment  behavior,  to  systems  that  aid  in  fault  diagnosis 
and  repair.  Application  domains  range  from  production  equipment  (e.g.,  clutch  assembly  machines),  to 
process  equipment  (e.g.,  turbine  generators),  to  high  technology  specialized  equipment  (e.g.,  fighter 
aircraft).  There  is  a  range  of  methodologies  employed,  as  well,  including  algorithmic  approaches  for  the 
preventive  maintenance  schedulers  to  expert  systems  for  fault  diagnosis  and  repair.  The  technologies 
employed  encompass  a  range  from  VAX  mini  computers  to  desktop  microcomputers  linked  to  video  disks. 
This  chapter  addresses  extant  approaches  to  job  aiding  in  maintenance,  the  prospects  for  using  emerging 
technologies  for  such  systems,  and  the  impact  of  emerging  technologies  on  human  performance, 
particularly  in  aviation  maintenance  applications.  This  section  also  calls  for  a  new  design  philosophy  in 
building  job  aids.  A  study  which  used  this  philosophy  and  compared  three  different  levels  of  aiding  on 
a  task  is  also  discussed.  Some  of  the  results  of  the  study  and  their  applicability  to  maintenance  job  aids 
are  presented. 

This  chapter  is  similar  to  a  previous  review  of  job  aids  (see  Chapter  5  of  Shepherd,  et  a].,  1991),  in  that 
many  of  the  systems  encountered  were  concerned  with  technological  developments,  rather  than 
performance  achievements.  Whereas  that  previous  work  identified  some  of  the  difficulties  with 
introducing  advanced  technology  job  aids  into  an  operational  environment,  this  discussion  addresses  some 
of  the  fundamental  problems  with  past  approaches  to  job  aids  and  presents  a  design  philosophy  which 
capitalizes  on  the  skills  and  abilities  of  the  operator  in  order  to  produce  a  combined  human-computer 
system  that  attains  increased  performance. 

3.1  SURVEY  OF  MAINTENANCE  JOB  AIDS 

A  survey  of  academic,  industrial,  and  popular  literature  revealed  a  wide  variety  of  approaches  to  building 
maintenance  job  aids  (see  Appendix).  These  differing  approaches  include  both  hardware  and 
methodological  considerations,  ranging  from  stand-alone,  automatic  scheduling  systems  to  portable, 
interactive  troubleshooting  systems.  The  hardware  aspects  are  addressed  first,  followed  by  a  discussion 
of  some  of  the  different  methods  used. 

3.1.1  Hardware  Employed 

The  following  systems  exemplify  different  hardware  approaches  used  for  maintenance  job  aids.  These 
systems  are  presented  in  order  of  increasing  sophistication. 

Policy  and  Hritz  ( 1987)  describe  an  expert  system  that  assists  in  troubleshooting  clutch  assembly  machines 
on  a  production  line.  Fault  lamps  above  the  machine  stations  indicate  which  stations  are  malfunctioning. 
A  technician  takes  a  maintenance  cart  to  the  malfunctioning  station.  The  cart  carries  a  two-button  control 
and  a  monitor  and  the  technician  connects  these  to  a  junction  box  at  the  station.  This  junction  box  links 
the  monitor  and  control  to  a  remote  computer  and  video  disk  player.  The  technician  uses  the  control  to 
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move  through  a  menu  system  to  specify  the  faulty  station.  The  computer  then  specifies  the  tests  to  be 
performed,  along  with  graphic  displays  of  the  equipment,  and  the  technician  enters  the  results  of  the  tests. 
In  this  way,  the  computer  guides  the  technician  through  houbleshooting  and  repairing  the  malfunctioning 
equipment. 

A  similar  system  developed  by  the  Electric  Power  Research  Institute  (EPRl)  also  uses  a  video  disk  player 
for  displaying  maintenance  information  and  procedures  for  gas-turbine  power  plants.  This  system  uses 
a  dual  processor  computer  system.  One  processor  manages  an  expert  system,  while  another  controls  a 
video  disk  player.  The  EPRI  system  also  uses  voice  recognition  and  synthesis  for  input  and  output, 
respectively. 

General  Motors  developed  an  expert  system  to  assist  in  vibration  analysis  of  production  machinery  (cf. 
"GM  unveils  ‘Charley’...").  Named  after  a  retiring  technician  with  many  years  of  experience,  ‘Charley’ 
was  intended  to  help  less  experienced  technicians  locate  parts  that  needed  repair  in  production  equipment 
with  rotating  components.  Charley  stores  a  signature  file  for  each  properly  operating  piece  of  equipment; 
technicians  record  the  vibration  signature  of  a  problematic  piece  of  equipment  with  a  special  data  recorder 
and  then  connect  the  recorder  to  a  Sun  workstation.  Charley  compares  the  newly  recorded  signature  with 
the  database  and  begins  diagnosing  the  problem.  Charley  guides  interactions,  may  ask  the  technician  for 
additional  information,  and  explains  its  troubleshooting  strategies.  Charley  can  also  be  used  as  a 
consultant  and  allow  a  technician  to  explore  ‘what  if  questions.  Finally,  Charley  is  also  used  to  train  new 
technicians.  The  emphasis  of  the  system  is  on  preventive  maintenance,  rather  than  repair  of  failed 
equipment. 

McDonnell  Douglas  developed  the  ‘Avionics  Integrated  Maintenance  Expert  System’  (AIMES)  for  use 
on  F/A-18  fighter  airaaft  (cf.  "McDonnell  Douglas  flight  tests...").  AIMES  is  a  self-contained  on-board 
box  which  contains  a  microprocessor  and  records  flight  avionics  data  on  a  cassette  for  later  analysis. 
Production  rules  detect  and  isolate  avionic  failures  at  the  electronic  card  level.  AIMES  generates  queries 
and  tests  based  on  data  and  concludes  whether  a  fault  is  present.  If  there  is  a  fault,  AIMES  supplies  the 
fault  data,  the  card  name,  and  the  reasoning  that  led  to  the  fault  isolation  conclusion. 

The  telecommunications  industry  is  a  large  user  of  advanced  technology  maintenance  aids,  particularly 
in  network  switch  and  cable  analysis  (cf.  "Expert  system  from  AT&T...").  The  ‘Automated  Cable 
Expertise’  system  runs  automatically  each  night  to  detect  trouble  spots  in  cables.  Upon  identifying  a 
problem,  it  reports  the  repair  history  of  the  area  and  suggests  corrective  action. 

3.1,2  Methods  Employed 

The  following  systems  exemplify  the  range  of  methodologies  employed  in  maintenance  job  aids.  These 
systems  are  presented  in  order  of  inaeasing  sophistication. 

Bertiiouex,  Lai,  and  Darjatmoko  (1989)  discuss  a  system  for  determining  daily  operations  for  a  wastewater 
treatment  plant.  This  system  is  billed  as  an  ‘expert  system’,  although  it  was  developed  using  standard 
spreadsheet  (Lotus  1-2-3)  and  database  software  (d-Base  III),  rather  than  one  of  the  many  production 
system  shells.  (Expert  systems  have  historically  been  written  using  production  rules  (if-then  clauses)  in 
one  of  many  languages  specifically  designed  for  that  purpose,  for  example  OPS5  or  LISP.  Popularization 
of  the  term  ‘expert  system’  has  led  to  decreasing  precision  of  use  of  it.) 
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‘Process  Diagnosis  System’  (PDS)  was  developed  by  the  Westinghouse  Research  and  Development  Center 
and  Carnegie  Mellon  University  for  maintenance  of  steam  generators.  PDS  is  a  condition  monitoring 
system  for  preventive  maintenance  in  order  to  alleviate  both  breakdown  maintenance  and  unnecessary 
maintenance.  The  system  is  designed  to  detect  deterioration  early  and  predict  the  duration  of  safe 
operation.  PDS  also  recommends  specific  preventive  maintenance  for  regularly  scheduled  down  times. 

Vanpelt  and  Ashe  (1989)  describe  the  ‘Plant  Radiological  Status’  (PRS)  system  for  nuclear  power  plants. 
The  PRS  system  presents  a  three  dimensional  model  of  the  power  station  and  equipment  so  that 
maintenance  teams  may  plan  maintenance  tasks  in  advance.  The  PRS  system  facilitates  access  to  and 
interpretation  of  radiological  conditions  by  identifying  hotspots  and  contaminated  areas,  as  well  as 
identifying  obstructions  and  available  workspace.  The  goals  of  the  PRS  system  are  to  reduce  maintenance 
time  and  radiation  exposure. 

Several  systems  for  supporting  operations  and  maintenance  were  reviewed  by  Bretz  (1990).  One  of  the 
systems  was  developed  by  Chubu  Electric  Power  Company  and  Mitsubishi  Heavy  Industries,  Ltd.  in  Japan. 
This  comprehensive  expert  system  assists  in  power  plant  boiler  failure  analysis  and  maintenance  planning. 
The  failure  diagnosis  reports  the  most  probable  causes  for  failure,  guidelines  for  inspection,  the  items  to 
be  investigated,  repair  methods,  and  suggested  preventive  maintenance.  The  maintenance  planning 
subsystem  aMtomatically  prepares  daily  repair  schedules,  a  work  estimation  plan,  and  work  specifications. 

The  distinction  is  sometimes  made  between  ‘deep’  and  ‘shallow’  knowledge  in  expert  systems.  The 
knowledge  typically  represented  in  production  systems  is  considered  shallow  knowledge  because  it 
contains  only  antecedent-consequent  relationships  without  any  information  as  to  why  one  thing  follows 
from  the  other.  Deep  knowledge,  on  the  other  hand,  captures  the  functional  and  causal  relationships 
between  the  components  of  the  object  or  system  being  model.  Atwood,  Brooks,  and  Radlinski  ( 1986)  call 
‘causal  models,’  which  use  components  functions  as  the  basis  for  their  reasoning,  the  next  generation  of 
expert  systems.  Clancy  (1987)  describes  a  system  for  diagnosing  switch  mode  power  supplies  which  uses 
a  model  of  the  component  level  of  the  electronics  for  its  diagnosis.  Whereas  one  can  test  for  signal 
presence  at  the  module  level  of  the  electronics,  the  component  level  is  concerned  with  the  way  in  which 
a  signal  changes  as  it  passes  through  the  components.  Finally,  a  system  developed  for  Britain’s  Central 
Electricity  Governing  Board  uses  a  model  of  the  cause  and  effect  relationships  inherent  in  turbine 
generators  for  diagnosis  and  maintenance  (see  "Expert  system  probes...").  This  expert  system  monitors 
and  analyzes  the  vibration  patterns  of  the  equipment  in  its  analysis. 

The  most  sophisticated  system  encountered  in  the  survey  is  the  ‘Testing  Operations  Provisioning 
Administration  System’  (TOP AS)  developed  by  AT&T.  Clancy  (1987)  describes  TOPAS  as  a  real-time, 
distributed,  multi-tasking  expert  system  for  switched  circuit  maintenance.  TOPAS  performs  trouble 
analysis,  localization,  and  referral  of  network  troubles.  Clancy  claims  that  TOPAS  "does  network 
maintenance  without  human  intervention  or  consultation"  (p.  103).  If  this  is  true,  then  TOPAS  is  not 
really  a  job  because  it  performs  the  job  itself. 


3.2  THE  USE  OF  ARTIFICIAL  INTELLIGENCE  IN  JOB  AIDS 

The  methods  and  design  philosophies  used  in  building  job  performance  aids  vary  with  the  designer(s). 
While  some  of  the  systems  surveyed  placed  the  technician  in  charge  of  the  troubleshooting  and 
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maintenance,  the  majority  of  the  approaches  relied  on  artificial  intelligence.  The  following  describes 
various  artificial  intelligence  approaches  and  their  impact  on  human  performance. 

3.2.1  Expert  Systems 

Expert  systems  typically  have  three  components;  a  rule  base,  a  knowledge  base,  and  an  inference  engine. 
The  rule  base  contains  the  problem  solving  strategies  of  an  expert  in  the  domain  for  which  the  system  was 
developed.  The  rule  base  is  made  up  of  production  rules  (if-then  clauses).  The  knowledge  base  contains 
the  history  and  the  current  data  of  the  object  under  consideration  (this  object  may  be  anything  from  an 
aircraft  engine  to  a  medical  patient).  The  inference  engine  is  responsible  for  determining  what  rules  get 
activated  and  when  the  system  has  solved  the  problem  or  is  at  an  impasse.  Expert  systems  are  typically 
written  in  a  programming  language  specifically  designed  for  such  use,  such  as  LISP  or  OPS5. 

Typically,  the  human  expert  is  not  the  person  who  builds  the  expert  system,  rather  he/she  interacts  with 
a  ‘knowledge  engineer’  who  is  responsible  for  extracting  the  expert’s  expertise.  One  difficulty  with  expert 
systems  has  frequently  been  referred  to  as  the  ‘knowledge  engineering  bottleneck’;  it  can  be  difficult  to 
access  and  program  the  knowledge  of  the  expert  into  the  expert  system.  For  instance,  the  exfxjrt  may  not 
even  be  aware  of  what  he/she  does  to  solve  a  particular  problem.  Furthermore,  it  is  impossible  to 
guarantee  that  the  rule  base  contains  all  of  the  knowledge  of  the  expert. 

3.2.2  Knowledge- Based  Systems 

Knowledge-based  systems  place  less  emphasis  on  production  rules  as  a  way  of  representing  knowledge, 
and  more  emphasis  on  using  a  large  database  of  information.  This  database  may  consist  of  information 
such  as  vibration  patterns  of  equipment,  as  in  Charley  discussed  above,  or  it  may  consist  of  typical 
hardware  configurations,  for  instance.  The  point  of  knowledge-based  systems  is  that  they  rely  on  a  large 
body  of  readily-available  information  for  the  bulk  of  their  processing. 

3.2J  Model-Based  Systems 

Model-based  systems  are  an  attempt  to  produce  more  robust  problem  solving  systems  by  relying  on  ‘deep’ 
representations  of  a  domain.  The  models  depend  on  a  description  of  the  functionality  and  relationships 
of  the  components  that  make  up  the  domain.  Mcxlcl-based  systems  are  concerned  with  not  only  how  a 
component  functions,  but  why  it  functions  that  way.  Developers  of  model-based  systems  believe  that 
these  systems  will  be  able  to  solve  novel  problems,  whereas  expert  .systems  can  only  solve  problems  with 
which  an  expert  is  familiar. 

3.3  HUMAN  PERFORMANCE  IMPLICATIONS  OF  ARTIFICIAL  INTELLIGENCE 
APPROACHES 

The  human  performance  implications  of  u.sing  an  artificial  intelligcnce-ba.sed  problem  solver  are  many. 
All  of  these  systems  revolve  around  the  ‘machine  expert’  paradigm,  in  which  the  computer  controls  all 
problem-solving  activities.  One  problem  with  the  machine  expert  paradigm  is  that  because  computers  do 
not  have  access  to  the  ‘world’,  they  must  rely  on  a  person  to  supply  all  relevant  data  about  the  world. 
Thus,  the  machine  expert  directs  tests  to  be  run  and  requesits  the  results  of  those  tests.  Based  on  these 
data,  the  computer  requests  more  information  or  reaches  a  conclusion,  ar.j  that  conclusion  may  be 
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erroneous.  In  the  words  of  one  cognitive  engineering  researcher,  the  human  is  reduced  to  a  "data  gatherer 
and  solution  filter"  for  the  machine. 

One  problem  associated  with  this  lack  of  environmental  access  is  that  the  person  may  have  knowledge  that 
the  computer  does  not.  Since  the  computer  directs  the  problem  solving,  it  may  never  ask  for  information 
that  may  be  critical  to  successfully  solving  the  problem.  Furthermore,  there  is  usually  no  provision  for 
the  operator  to  volunteer  such  information.  The  person  may  even  have  different  goals  than  the  machine 
or  may  not  know  what  the  machine’s  goals  are  when  it  is  attempting  to  solve  a  particular  problem. 
Additional  difficulties  arise  when  the  human  operator  accidentally  enters  the  wrong  data  or  when  he/she 
misinterprets  a  request  from  the  computer.  Suchman  (1987)  discusses  the  problems  of  human  machine 
communication  at  length. 

Probably  the  biggest  problem  associated  with  expert  systems  is  that  they  are  brittle.  As  mentioned  above, 
expert  systems  can  only  solve  problems  that  the  human  expert  has  seen  or  remembers  to  discuss  with  the 
knowledge  engineer.  People  (either  experts  or  expert  system  designers)  simply  cannot  anticipate  all  of 
the  environmental  variability  encountered  in  the  world.  This  leads  to  the  tragic  irony  of  such  systems: 
expert  systems  are  most  needed  when  a  problem  is  difficult,  and  that  is  precisely  when  the  expert  systems 
fail.  The  upshot  is  that  the  human  operator  is  left  to  solve  a  difficult  problem  without  the  benefit  of 
having  developed  expertise  through  solving  other  problems,  because  those  were  handled  by  the  expert 
system! 

All  of  these  problems  and  more  arose  in  a  study  by  Roth,  Bennett,  and  Woods  (1987),  in  which  the 
authors  observed  technicians  using  an  expert  system  to  troubleshoot  an  electro-mechanical  device.  One 
of  the  major  findings  of  the  study  was  that  only  those  technicians  who  were  actively  involved  in  the 
problem  solving  process  and  performed  activities  beyond  those  requested  by  the  expert  system  were  able 
to  complete  the  tasks.  The  technicians  who  passively  performed  only  those  activities  requested  by  the 
expert  system  were  unable  to  reach  solutions  on  any  but  the  most  trivial  tasks. 

The  above  should  not  be  interpreted  as  a  condemnation  of  all  u.scs  of  artificial  intelligence  techniques, 
however.  Indeed,  artificial  intelligence  has  greatly  advanced  our  understanding  of  the  capabilities,  as  well 
as  the  limitations,  of  computational  tools.  Prudent  use  of  such  techniques  can  greatly  enhance  the  ability 
of  a  cognitive  engineer  to  provide  operators  with  powerful  problem  solving  tools. 

3.4  EMERGING  TECHNOLOGIES 

Continued  advances  in  hardware  and  software  technologies  will  further  increase  the  cognitive  engineer’s 
design  repertoire.  Indeed,  there  are  many  emerging  technologies  that  could  be  profitably  used  in 
maintenance  job  aids.  Advances  in  computer  hardware,  display  hardware,  and  t)bject  modeling  all  have 
great  potential  to  improve  job  aiding  capabilities.  Each  of  these  is  discussed  below. 

3.4.1  Advances  in  Computer  Hardware 

As  computer  hardware  has  become  smaller  and  more  powerful,  there  has  been  a  progression  to  smaller, 
more  portable  job  aids.  Whereas  earlier  job  aids  ran  on  minicomputers,  then  workstations  and  pcr.sonal 
computers,  newer  job  aids  are  being  designed  using  laptops.  There  is  no  reason  to  believe  that  the  laptop 
computer  is  the  smallest,  lightest  computer  that  will  be  developed,  however.  Indeed,  the  NCR  NotePad 
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has  recently  been  introduced.  This  computer  is  pen-based;  that  is,  all  input  is  performed  via  a  pen  stylus, 
rather  than  through  a  keyboard  or  mouse.  The  NotePad  is  light  enough  that  it  can  be  easily  held  in  one 
hand,  which  greatly  facilitates  taking  it  to  the  maintenance  site.  The  NotePad  is  relatively  quick,  it  has 
reasonably  large  storage  capacity,  and  it  has  limited  handwriting  recognition  abilities. 

An  aviation  industry  working  group  is  currently  defining  the  standards  for  a  ‘Portable  Maintenance  Access 
Terminal’  (PMAT)  for  use  in  commercial  aviation.  As  currently  conceived,  the  PMAT  would  connect  to 
the  ‘Onboard  Maintenance  Systems’  of  current  aircraft  and  would  be  used  for  troubleshooting.  Because 
the  emphasis  is  on  portability,  it  is  likely  that  something  similar  to  the  NotePad  or  a  standard  laptop 
computer  will  be  specified. 

Another  emerging  hardware  technology  is  the  use  of  ‘built-in  test  equipment’  (BITE)  in  engineered 
systems,  no  doubt  due  in  part  to  the  widespread  use  of  microprocessors.  BITE  likely  does  not  eliminate 
the  maintenance  technician,  however,  because  it  may  be  difficult  to  implement  such  equipment  in 
mechanical  systems  or  in  very  complex  systems.  Indeed,  BITE  may  introduce  additional  problems  for 
maintenance  people  because  there  is  a  lack  of  standardization  on  how  BITE  should  opierate;  thus,  there 
may  be  confusion  when  dealing  with  similar,  but  different,  BITE.  Further  complications  may  arise  due 
to  issues  of  granularity  in  BITE;  BITE  may  simply  indicate  that  a  piece  of  equipment  is  not  functioning 
properly,  without  indicating  the  specific  nature  of  the  malfunction  or  without  indicating  which  component 
must  be  repaired  or  replaced.  Another  issue  is:  What  happens  when  the  BITE  malfunctions? 

3.4.2  Advances  in  Display  Hardware 

One  of  the  surveyed  systems  used  a  personal  computer  to  control  a  slide  projector  for  displaying 
maintenance  graphics.  Several  of  the  systems  used  a  computer-controlled  video  disk  for  such  displays. 
With  the  advent  of  digital  cameras  and  compact  disc-interactive  (GDI)  technology,  systems  with  higher 
fidelity  and  portability  can  be  achieved.  Appropriately  designed  GDI  systems  could  store  many  views  of 
the  object(s)  being  serviced,  as  well  as  maintenance  procedures  and  information.  Indeed,  what  graphics 
were  displayed  would  depend  on  the  fault  manifestations.  Furthermore,  well-designed  GDI  systems  would 
allow  the  technician  to  troubleshoot  by  hypothesizing  a  failed  component  and  watching  how  a  simulation 
of  the  system  performed.  Similarly,  the  technician  could  replace  a  component  in  the  simulation  and  see 
the  results.  In  this  manner,  the  technician  could  develop  expertise  more  quickly  than  learning  on-the-job 
(because  the  technician  would  have  control  over  what  aspects  he  was  learning,  rather  than  relying  on 
whatever  malfiinction  happened  to  occur). 

3.43  Advances  in  Object  Modeling 

An  extension  of  the  three-dimensional  model  discussed  above  is  virtual  reality.  Virtual  reality  has 
received  a  lot  of  attention  as  a  result  of  the  Defense  Advanced  Research  Project  Agency’s  development 
of  the  ‘Pilot’s  Associate  Program’  and  consists  of  replacing  an  operator’s  view  of  the  ‘real  world’  with 
a  simulated  view  of  that  world.  Thus,  real  world  objects  are  replaced  with  simulations  of  those  objects. 
One  possible  use  of  virtual  reality  would  be  to  allow  the  maintenance  technician  to  ‘stand’  inside  a  device, 
such  as  an  engine,  and  watch  how  it  functions,  both  normally  and  with  failed  components.  The  technician 
could  also  see  the  effects  of  replacing  components,  similar  to  the  GDI  system  above,  but  with  the  benefit 
of  observing  the  effects  more  directly.  As  with  GDI,  the  technician  need  not  replace  the  actual  system 
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components,  but  may  replace  components  in  the  simulation  of  that  system.  The  uses  of  virtual  reality 
appear  to  be  limited  only  by  the  job  aid  designer’s  imagination. 

3.5  HUMAN  PERFORMANCE  IMPLICATIONS  OF  EMERGING  TECHNOLOGIES 

While  many  past  approaches  to  job  performance  aids  sought  to  replace  human  expertise  with  machine 
expertise,  there  is  a  growing  appreciation  for  the  importance  of  human  skill.  The  machine  expert  paradigm 
sought  to  overcome  human  information  processing  ‘limitations’  with  a  computer  prosthesis.  However, 
even  computers  are  limited  resource  processors.  A  more  enlightened  approach  is  to  view  computers  as 
tools  to  amplify  human  capabilities,  not  overcome  limitations.  In  this  sense,  computers  can  be  seen  to  be 
like  other  tools,  such  as  telescopes  or  automobiles:  they  are  instruments  which  provide  additional 
resources  for  achieving  our  needs  and  desires.  Woods  and  Roth  (1988)  discussed  the  above  issues  and 
addressed  many  more  cognitive  engineering  issues  inherent  to  developing  systems  that  have  powerful 
computational  abilities. 

Technology  is  not  a  panacea;  each  new  technology  brings  with  it  significant  drawbacks,  as  well  as 
benefits.  The  challenge  to  designers  is  to  use  emerging  technologies  to  build  cooperative  systems,  in 
which  both  the  human  and  the  computer  are  actively  involved  in  the  problem  solving  process.  Humans 
can  no  longer  be  regarded  as  passive  ‘users’  of  technology,  but  as  competent  domain  practitioners  with 
knowledge  and  abilities  which  are  difficult  to  replace.  The  following  section  discusses  a  study  which 
addressed  just  such  issues. 

3.6  A  STUDY  OF  HUMAN  PERFORMANCE  WITH  A  COOPERATIVE  SYSTEM 

A  study  which  addressed  some  of  the  human  performance  issues  discussed  above  was  carried  out  as  part 
of  the  author’s  graduate  program  (Layton,  1992).  This  study  compared  three  different  levels  of  computer 
support  on  the  basis  of  their  effects  on  human  performance.  Although  the  domain  for  which  the  systems 
were  developed  was  enroute  flight  planning,  the  general  principles  behind  the  alternative  designs  can  be 
applied  to  developing  aviation  maintenance  aids,  as  well.  The  following  is  a  discussion  of  enroute  flight 
planning,  the  design  concepts  behind  the  three  levels  of  computer  support,  the  method  employed  for 
comparing  the  various  systems,  the  general  outcomes  of  the  study,  and  the  implications  of  those  outcomes 
for  developing  aircraft  maintenance  job  aids. 

3.6.1  Enroute  Flight  Planning 

Enroute  flight  planning  consists  of  modifying  the  flight  plan  of  an  airborne  aircraft  in  response  to  changes 
in  the  capabilities  of  the  aircraft,  to  crew  or  passenger  emergencies,  to  changes  in  weather  conditions, 
and/or  to  problems  at  the  destination  airport.  The  study  focused  on  flight  plan  adaptation  in  response  to 
changes  in  weather  conditions.  From  a  pilot’s  perspective,  the  components  important  to  enroute  flight 
planning  include  the  airplane,  possible  flight  routes,  weather  conditions,  and  airline  company  dispatchers. 
The  pilot  is  concerned  with  getting  from  a  given  origin  to  a  given  destination  on  time,  with  a  minimum 
of  fuel  consumed,  while  maintaining  flight  safety.  He/she  must  consider  what  routes  to  take  (these  routes 
consist  of  waypoints,  or  navigational  points,  and  jet  routes,  the  so-called  "highways  in  the  sky"),  what 
altitudes  to  fly,  what  weather  to  avoid,  and  the  ever-changing  capabilities  of  the  aircraft  (eg.  the  weight 
of  the  plane  decreases  with  fuel  consumption;  the  lighter  the  plane,  the  higher  it  can  fly,  within  limits). 
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The  initial  flight  plan  is  rarely  followed  exactly,  due  to  unforeseen  events  occurring  while  enroute. 
Indeed,  minor  changes  in  flight  plans  are  frequently  made  and  major  changes  are  fairly  common.  These 
amendments  to  the  original  result  from  the  dynamic,  unpredictable  nature  of  the  ‘world’  in  which  the  plans 
are  carried  out.  Weather  patterns  do  not  always  develop  as  predicted,  resulting  in  unexpected  areas  of 
turbulence,  less  favorable  winds,  or  storms  that  must  be  avoided.  Air  traffic  congestion  may  delay  take-off 
or  restrict  the  plane  to  lower-than-planned  altitudes.  Airport  or  runway  closures  can  cause  major 
disruptions,  not  just  for  one  aircraft,  but  for  everyone  planning  on  landing  at  that  airport.  Mechanical 
failures,  medical  emergencies,  or  other  critical  problems  may  delay  take-off  or  may  force  an  airborne  plane 
to  divert  to  a  nearby  airport. 

Furthermore,  there  are  several  constraints  on  the  flight  plans  that  can  be  developed.  Planes  must  maintain 
a  certain  separation  distance  between  each  other  and  between  thunderstorm  cells,  as  specified  in  the 
Federal  Air  Regulations.  Planes  must  fly  along  the  jet  routes.  They  are  also  limited  to  certain  altitudes. 
Over  the  continental  United  States,  for  example,  33,000  feet  is  an  ’eastbound  only’  altitude.  There  are 
also  physical  limitations;  the  plane  can’t  fly  if  it  is  out  of  fuel  and  it  can’t  land  at  an  airport  with  runways 
that  are  too  short.  Some  of  these  constraints  are  actually  ’soft’,  in  that  they  may  be  violated  in  some 
circumstances.  If,  for  instance,  there  is  no  eastbound  traffic.  Air  Traffic  Control  (ATC)  may  allow  a  plane 
to  fly  west  at  an  ’eastbound  only’  altitude.  Similarly,  ATC  may  approve  a  vector  that  deviates  from  the 
jet  routes  in  order  to  avoid  a  storm  or  to  save  fuel. 

3.6,2  System  Design  Concepts 

It  is  clear  that  enroute  flight  planning  is  a  complex  activity,  but  it  is  not  clear  how  humans  deal  with  these 
complexities  or  how  one  might  program  a  computer  to  choose  the  ’optimum’  solution  to  any  given 
problem.  For  instance,  how  does  one  make  tradeoffs  between  fuel  conservation,  flight  safety,  and  prompt 
arrival  at  the  destination?  Because  pilots  make  such  tradeoffs  on  a  routine  basis,  one  goal  of  the  study 
was  to  develop  a  system  to  support  the  pilots  in  making  such  decisions.  There  is  a  heavy  emphasis, 
therefore,  on  allowing  the  pilots  to  explore  "what  if’  types  of  questions  so  that  they  could  gain  feedback 
on  the  impact  of  a  planning  decision  on  flight  parameters. 

The  three  levels  of  computer  support  corresponded  to  successively  greater  flight  planning  power. 
Common  to  all  three  systems  were:  1.  a  map  display  which  consisted  of  the  continental  United  States, 
the  aircraft,  and  flight  routes;  2.  a  representation  of  a  flight  log,  which  included  the  flight  route  and 
altitudes;  and,  3.  a  display  of  flight  parameters.  These  three  items  were  displayed  on  two  monitors. 
Figure  3.1  depicts  the  map  displays  and  controls,  and  Figure  3.2  depicts  the  flight  log  display  and 
controls  and  the  flight  parameter  display.  The  pilot  could  elect  to  display  weather  data,  waypoints,  and 
jet  routes  on  the  map  display.  The  lowest  level  of  enroute  flight  planning  support  provided  the  pilot  with 
the  ability  to  sketch  proposed  flight  plans  on  the  map,  in  accordance  with  the  waypoint  and  jet  route 
structure.  The  latter  condition  required  a  pilot  to  sketch  routes  one  waypoint  at  a  time.  Once  the  pilot 
completed  a  proposed  flight  plan,  in  terms  of  geographic  location,  the  computer  responded  with  various 
flight  parameters,  such  as  time  of  arrival  and  fuel  remaining  at  the  destination.  'The  computer  also 
indicated  whether  the  flight  was  predicted  to  encounter  any  turbulence  and  the  severity  of  that  turbulence. 
The  computer  also  proposed  the  most  fuel  efficient  vertical  flight  profile  for  the  proposed  route.  This 
form  of  support  encouraged  the  pilots  to  propose  options  and  see  their  effects  on  flight  parameters.  This 
form  of  support  is  referred  to  as  the  ‘sketching  only’  system. 
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The  next  level  of  computer  support  incorporated  the  sketching  form  of  interaction,  but  also  included  a 
method  for  placing  constraints  on  a  desired  solution  and  allowing  the  computer  to  propose  a  solution 
which  satisfied  those  constraints.  For  instance,  the  pilot  could  place  limits  on  the  maximum  severity  of 
turbulence  and  precipitation  encountered,  and  could  specify  the  desired  destination.  The  computer  would 
then  perform  a  search  of  the  data  and 
solution  spaces  and  propose  a  route 
that  satisfied  the  pilot’s  constraints 
while  minimizing  fuel  consumption. 

This  proposed  route  would  include 
both  the  geographic  route  and  the 
vertical  profile,  along  with  its 
associated  flight  parameters.  This  form 
of  flight  planning  causes  the  pilot  to 
plan  at  a  more  abstract  level  than  the 
sketching  form  of  interaction,  because 
the  pilot  is  able  to  think  about  the 
characteristics  of  a  desired  solution 
while  the  computer  handles  the  lower 
level  details  of  specific  routings. 

Using  the  sketching  tool,  the  pilot  was 
free  to  modify  the  route  proposed  by 

the  computer  and  note  the  impact  of  .  .  ^ 

such  changes  on  the  flight  parameters.  Figu*"®  3.1  Left  Monitor  Displays  and  Controls 


This  second  level  of  planning  can  be 
roughly  construed  to  be  a  form  of 
consultation  system  because  the 
computer  can  be  asked  for  its  advice  on 
a  problem;  it  is  referred  to  as  the  ‘route 
constraints  and  sketching’  system. 

The  highest  level  of  support 
corresponds  to  an  expert  system  that 
automatically  solves  a  problem  as  soon 
as  it  is  detected;  upon  loading  the 
scenario  informafion,  the  computer 
would  propose  a  solution  which 
minimized  fuel  consumption  and 
satisfied  the  constraints  of  encountering 
no  turbulence  and  no  precipitation,  as 
well  as  arriving  at  the  planned 
destination.  As  in  the  previous  level  of 
support,  the  computer  would  propose 


both  the  geographic  route  and  altitude  profile,  along  with  the  corresponding  flight  parameters.  If  desired, 
the  pilot  could  also  request  a  solution  from  the  computer  based  on  different  constraints,  and  he  could 


sketch  his  own  solutions. 
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3.6J  Study  Method 

Thirty  male  commercial  airline  pilots  were  randomly  assigned  to  one  of  three  treatment  conditions, 
wherein  each  condition  consisted  of  one  of  the  three  forms  of  computer  support  described  above.  There 
were  ten  subjects  in  each  condition.  Each  pilot  was  trained  for  approximately  one  hour  on  his  system 
prior  to  solving  four  enroute  flight  planning  cases.  Each  case  consisted  of  a  planned  flight  that  was 
disrupted  because  of  a  change  in  weather  conditions.  The  task  for  the  pilot  was  to  decide  what  to  do  in 
each  situation.  All  of  the  pilots  solved  the  four  cases  in  the  same  order.  It  took  approximately  an  hour 
and  a  half  to  solve  the  four  cases. 

3.6.4  Study  Results 

Each  of  the  four  cases  provided  some  interesting  insights  into  the  influences  of  computer  tools  on  human 
behaviors.  The  overriding  results  of  each  of  the  four  cases  are  discussed  below. 

3.6.4.1  Case  1  General  Results 

In  the  first  case,  most  of  the  subjects  in  the  ‘route  constraints  and  sketching’  and  the  ‘automatic  route 
constraints,  route  constraints,  and  sketching’  conditions  chose  to  fly  the  computer-suggested  route  (as 
expected).  However,  the  ‘sketching  only’  subjects  tended  to  choose  routes  that  were  more  robust;  that 
is,  these  subjects  put  more  distance  between  the  aircraft  and  the  storm.  These  subjects  commented  that 
they  would  like  to  have  more  distance  from  the  storm  than  afforded  by  a  more  direct  route  (such  as  the 
one  suggested  by  the  computer  in  the  other  two  treatment  conditions).  Furthermore,  the  ‘sketching  only’ 
subjects  were  more  apt  to  explore  multiple  routes  and  multiple  types  of  routes,  than  were  the  subjects  in 
the  other  two  groups.  These  results  suggest  that  the  sketching  form  of  interaction  caused  the  subjects  to 
consider  the  data  more  carefully  than  did  the  route  constraints  tool.  One  reason  for  this  result  is  that  the 
sketching  tool  gave  the  subjects  the  opportunity  to  consider  the  relationships  of  various  route  options  and 
the  weather  at  several  points  and  to  consider  the  robustness  of  those  options  given  the  uncertainties 
associated  with  weather.  The  constraints  tool,  on  the  other  hand,  did  not  encourage  such  behavior,  and, 
indeed,  the  subjects  using  that  tool  may  have  been  under  the  impression  that  the  computer  was  considering 
the  robustness  of  routes,  when  in  fact  it  was  not.  If  the  sketching  tool  encouraged  more  careful 
examination  of  the  data  than  did  the  constraints  tool,  and  this  behavior  persisted,  one  could  imagine 
situations  wherein  the  constraints  tool  could  lead  to  bad  decisions. 

3.6.4.2  Case  2  General  Results 

While  Case  1  provided  evidence  for  the  benefits  of  tools  that  make  the  operator  the  sole  decision  maker. 
Case  2  provided  evidence  to  the  contrary.  In  Case  2,  the  ‘sketching  only’  subjects  had  significant 
difficulty,  as  a  group,  in  searching  the  relatively  large  data  and  solution  spaces.  Many  of  the  routes 
explored  by  these  subjects  passed  through  strong  turbulence.  Indeed,  four  of  these  ten  subjects  chose 
deviations  that  exacted  a  high  fuel  consumption  cost,  either  because  they  could  not  find  a  more  efficient 
route  around/through  the  weather  or  because  they  did  not  examine  wind  data  which  would  have  indicated 
that  their  chosen  route  encountered  strong  head  winds.  By  contrast,  the  subjects  in  the  ‘route  consUaints 
and  sketching’  and  ‘automatic  route  constraints,  route  constraints,  and  sketching’  groups  successfully  used 
the  computer  to  rapidly  find  a  fuel  efficient  deviation  that  avoided  all  of  the  weather.  Furthermore,  nearly 
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all  of  the  subjects  who  chose  an  inefficient  deviation  later  stated  that  they  preferred  the  more  efficient 
deviation  suggested  by  the  computer  to  the  other  groups. 

3.6.43  Case  3  General  Results 

As  noted  in  the  discussion  of  Case  1,  the  ‘sketching  only’  subjects  chose  rather  different  solutions  than 
did  the  ‘route  constraints  and  sketching’  and  the  ‘automatic  route  constraints,  etc.’  subjects.  Furthermore, 
it  was  hypothesized  that  the  ‘sketching  only’  subjects  were  more  involved  in  the  problem  solving  process 
than  were  the  subjects  in  the  other  two  groups.  The  third  case  was  designed  to  address  the  issues  related 
to  what  happens  when  the  automatic  tools  suggest  questionable  solutions:  Does  the  operator  recognize 
that  the  solution  may  not  be  appropriate?  Assuming  the  operator  does  recognize  that  the  solution  is 
inappropriate,  can  he  readily  come  up  with  a  better  solution? 

In  Case  3,  the  computer  suggested  two  different  routes  in  the  ‘route  constraints  and  sketching’  and 
‘automatic  route  constraints,  etc.’  conditions,  depending  upon  the  constraints  placed  on  it.  One  deviation 
passed  between  two  large  thunderstorm  cells  of  a  volatile  storm,  which  is  a  risky  practice,  at  best;  this 
route  was  suggested  on  the  basis  of  no  turbulence  and  no  precipitation.  The  other  route  avoided  the  bulk 
of  the  weather,  at  the  cost  of  slightly  higher  fuel  consumption  and  a  small  amount  of  turbulence;  this  route 
was  suggested  on  the  basis  of  light  chop  (or  greater)  turbulence  and  light  (or  heavier)  precipitation.  The 
trend  in  this  case  was  for  the  ‘route  constraints  and  sketching’  and  the  ‘automatic  route  constraints,  route 
constraints,  and  sketching’  subjects  to  choose  the  first  route  more  frequently  than  the  ‘sketching  only’ 
subjects.  If  these  subjects  had  not  examined  both  routes,  then  it  would  suggest  that  these  subjects  were 
simply  over-reliant  on  the  computer.  However,  several  of  the  subjects  in  the  ‘route  constraints  and 
sketching’  and  ‘automatic  route  constraints,  etc.’  groups  examined  both  routes  before  choosing  the  more 
risky  route;  thus,  these  subjects  chose  a  risky  route  despite  evidence  that  it  may  have  been  a  poor  choice 
and  that  a  better  option  existed.  These  subjects  nearly  unanimously  changed  their  minds  when  later 
questioned  about  their  decisions. 

With  few  exceptions,  the  ‘sketching  only’  subjects  planned  very  conservative  deviations  that  completely 
avoided  the  weather.  However,  the  ‘sketching  only’  subjects  had  considerable  difficulty  in  finding 
acceptable  deviations.  In  fact,  one  subject  chose  a  deviation  that  was  predicted  to  cut  into  his  required 
landing  fuel  reserves  prior  to  arrival  at  the  destination.  Thus,  even  though  the  ‘sketching  only’  subjects 
may  have  considered  the  data  very  carefully,  the  problem  was  sufficiently  complex  that  they  would  have 
benefitted  from  some  computer  assistance. 

3.6.4.4  Case  4  General  Results 

Case  4  provided  some  interesting  results  with  regard  to  individual  differences  and  with  regard  to  the 
influence  of  computer  recommendations.  The  ‘sketching  only’  and  ‘route  constraints  and  sketching’ 
subjects  were  nearly  evenly  divided  between  a  fuel  efficient  deviation  and  a  robust  deviation.  When  asked 
about  his  decision,  one  of  the  ‘sketching  only’  subjects  made  the  comment  that  the  decision  depended  on 
the  person’s  role  in  flying  the  aircraft  at  the  time:  if  the  captain  were  flying  that  leg,  he  would  go  one 
way  so  that  he  could  look  at  the  storm,  but  if  the  first  officer  were  flying  that  leg,  he’d  go  the  other  way 
around  so  that  he  could  see  the  storm.  Obviously  this  is  an  extreme  example,  but  it  underscores  the  role 
of  individual  differences  in  decision  making. 
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Unlike  the  subjects  in  the  other  two  groups,  the  ‘automatic  route  constraints,  route  constraints,  and 
sketching’  subjects,  were  more  likely  to  choose  the  computer-suggested,  economical  route,  even  when  they 
had  explored  both  routes.  Combined  with  the  results  of  Case  this  result  suggests  that  the  computer 
exerts  a  strong  influence  on  decision  making  when  it  recommends  a  solution  at  the  onset  of  a  problem. 

3.6.5  Study  Conclusions 

The  goal  of  the  research  was  not  to  determine  which  particular  version  of  an  enroute  llight  planning  tcxil 
resulted  in  the  best  human  performance.  Rather,  one  goal  was  to  see  how  human  behaviors  were 
influenced  by  the  tools  available.  Subjects  who  had  multiple  tools  available  to  them  (the  ‘route  constraints 
and  sketching’  subjects  and  the  ‘automatic  route  constraints,  route  constraint.s.  and  sketching’  subjects) 
were  able  to  use  them  to  develop  alternative  plans,  in  fact,  there  were  many  instances  in  which  the 
solution  recommended  by  the  computer  did  not  meet  the  needs  of  the  pilots,  so  the  pilots  developed  their 
own  plans  through  sketching.  Thus,  not  only  is  there  a  need  for  tools  that  allow  the  operator  to  go  beyond 
a  computer’s  solution,  but  there  is  a  need  to  support  individual  differences,  as  well. 

The  subjects  who  had  only  the  sketching  tool  available  to  them  closely  examined  the  available  data.  As 
a  result,  these  subjects  often  planned  robust  deviations  that  would  not  need  to  be  altered  if  there  were 
further  changes  in  the  weather.  Where  these  subjects  ran  into  difficulties,  however,  was  in  situations  in 
which  there  were  a  lot  of  potential  solutions  and  there  was  a  large  amount  of  data.  In  such  situations, 
these  subjects  had  trouble  finding  appropriate  solutions.  Indeed,  some  of  these  subjects  made  poor 
decisions  because  of  the.se  difficulties.  The  subjects  who  had  .some  form  of  computer  assistance  were  able 
to  more  efficiently  search  these  spaces,  but  with  some  costs. 

The  tool  that  automatically  suggested  a  solution  to  the  problem  as  soon  as  it  was  detected  did  not 
encourage  the  subjects  to  closely  examine  the  data.  While  this  fact  did  not  cause  problems  in  some  cases, 
it  clearly  did  lead  to  bad  decisions  in  others.  Furthermore,  the  automatic  tixil’s  influence  on  decision 
making  went  beyond  simple  over-reliance  to  the  point  where  it  shifted  attention  from  data  which  were 
important  to  making  a  good  decision. 

3.6.6  Implications  for  Maintenance  Job  Aids 

The  conclusions  outlined  above  can  be  readily  applied  to  developing  maintenance  Job  aids.  For  instance, 
one  of  the  conclusions  is  that  there  is  a  need  for  tools  that  allow  an  operator  to  go  beyond  a  computer’s 
solution.  As  discussed  above,  particularly  with  regard  to  Cxse  ?<,  and  as  discussed  by  Roth.  Bennett,  and 
Woods  (1987)  and  Suchman  (1987),  operators  frequently  have  knowledge  or  information  which  is  not 
available  to  the  computer,  but  which  is  critical  to  making  a  g(H>d  decision.  By  giving  the  authority  and 
responsibility  for  decision  making  to  the  operator,  and  by  providing  a  tool  which  supports  the  operators 
activities  (rather  than  the  other  way  around),  the  operator  is  free  to  explore  solutions  tliat  may  not  have 
been  designed  into  a  machine  expx;rt. 

Another  conclusion  reached  by  the  above  study  was  that  the  form  of  tool  that  required  a  person  to  make 
a  series  of  decisions  (the  sketching  tool)  encouraged  the  operator  to  think  hard  about  the  problem  and  to 
consider  the  available  data  at  a  deeper  level,  than  did  the  form  of  tool  that  encouraged  the  operator  to 
make  a  single  ‘yes’  or  ‘no’  decision  (the  automatic  route  constraints  tiud).  In  this  regard,  the  conclusion 
supports  the  notion  that  designers  need  to  "keep  the  person  in  the  loop". 
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However,  another  conclusion  of  the  above  study  was  that  "keeping  the  person  in  the  loop"  did  not  provide 
adequate  support  in  some  situations.  Indeed,  in  some  of  the  cases  (such  as  Cases  2  and  3)  some  of  the 
operators  were  simply  unable  to  find  adequate  solutions  on  their  own.  These  operators  could  have  used 
some  help  from  a  computer  in  exploring  solution  possibilities.  In  such  situations  this  is  rarely  a  reflection 
of  human  ‘limitations’,  rather  it  is  an  indication  of  the  difficulty  of  the  problem.  In  maintenance,  for 
instance,  diagnosing  multiple,  interacting  faults  is  a  difficult  problem.  One  symptom  may  be  characteristic 
of  several  faults,  or  one  fault  may  mask  the  presence  of  another.  A  tool  which  helps  to  focus  the 
diagnostician’s  attention  and  eliminate  false  leads  would  be  very  beneficial. 

Finally,  it  is  important  to  realize  that  each  person  has  a  different  style  of  decision  making;  two  people  who 
complete  the  same  training  course  on  a  given  method  for  dealing  with  a  problem  may  use  slightly  different 
approaches.  Such  differences  are  likely  to  increase  with  experience  as  each  person  learns  methods  that 
consistently  work  for  him/her.  Indeed,  experts  often  use  several  different  approaches  to  solving  truly 
difficult  problems  because  each  approach  has  unique  limitations  as  well  as  unique  benefits.  For  instance, 
knowledge  of  thermodynamics  may  help  localize  a  fault  to  a  heat  exchanger,  but  knowledge  of  circuits 
may  lead  one  to  test  the  power  supply  to  the  heat  exchanger,  as  well.  Thus,  tools  need  to  be  flexible  to 
support  such  individual  differences,  rather  than  use  a  single,  lockstep  approach,  as  in  the  case  of  ‘expert’ 
systems.  (Note  that  although  some  expert  systems  do  incorporate  the  observable  components  of  such 
methods,  they  do  not  allow  the  operator  direct  access  to  those  methods.  Because  the  knowledge  and 
capabilities  of  such  systems  are  necessarily  incomplete,  the  systems  are  ‘brittle’  in  the  face  of  difficult 
problems,  as  discussed  above  in  Section  3.4) 

3.7  RESEARCH  AND  DEVELOPMENT  PLAN 

The  above  discussion  points  to  the  challenge  for  cognitive  engineers  involved  in  designing  maintenance 
job  aids;  build  systems  that  capitalize  on  both  human  strengths  and  computer  strengths  so  that  task 
performance  is  improved.  As  outlined  above,  because  of  relatively  recent  advances  in  hardware  and 
software,  it  is  possible  to  use  sophisticated  computational  techniques  (eg.  cooperative  system  techniques) 
to  develop  real  time,  computer-based  Job  aids  for  a  wide  range  of  technical  tasks.  Furthermore,  hardware 
like  the  NCR  NotePad  will  make  it  easier  for  people  without  previous  computer  training  to  use  such  job 
aids.  We  are  working  with  the  Flight  Standards  Service  of  the  FAA  to  develop  a  Portable  Performance 
Support  System  (PPSS)  to  aid  Aviation  Safety  laspectors  in  their  daily  activities.  The  initial  focus  of  this 
effort  is  on  the  tasks  performed  by  Airworthine.ss  (maintenance)  Inspectors,  particularly  the  Ramp 
Inspections  task.  Inspectors  need  access  to  many  of  the  same  types  of  information  that  maintenance 
technicians  use.  Inspectors  must  also  document  their  activities  and  the  outcomes  of  tho.se  activities.  We 
are  taking  a  three  Phase  approach  to  developing  a  PPSS;  Phxse  I.  already  underway,  will  identify  a 
prospective  task  and  perform  an  information  needs  analysis  for  that  task;  during  Phase  II  we  will  design 
and  develop  a  prototype  PPSS  for  the  task;  and  we  will  create  a  plan  for  the  development  of  the  prototype 
into  a  fully  functional  system  during  Phase  III.  Each  of  these  phases  is  described  more  fully  below. 

3.7.1  Phase  I:  Problem  Definition  and  Information  Needs  Analysis  for  Aviation  Safety  Inspectors 

Goal:  Identify  a  typical  task  for  which  a  computer-based  job  aid  is  an  appropriate  application  and 
conduct  appropriate  information  needs  analysis  to  define  the  work  enviromnent  and  information  needs  for 
aviation  safety  personnel. 
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We  are  working  with  the  personnel  of  the  Fort  Lauderdale  Flight  Standards  District  Office  (FSDO)  to 
help  identify  an  appropriate  task  for  computer-based  job  aiding.  Such  a  task  should  be  one  which  is 
typical  for  the  personnel,  but  which  may  require  some  experience  to  attain  proficiency.  This  Phase  of  the 
research  is  an  ongoing  process  continued  throughout  the  life  of  the  project. 

The  task  initially  proposed  to  the  Fort  Lauderdale  FSDO  is  one  of  an  Airworthiness  Inspector  performing 
Ramp  Inspections.  Ramp  Inspections  are  used  to  verify  aircraft  airworthiness  just  prior  to  a  planned  flight. 
The  inspectors  walk  around  the  aircraft,  identifying  problem  areas  and  documenting  those  problems  on 
a  Program  Tracking  and  Reporting  Subsystem  (PTRS)  form  (if  an  aircraft  meets  safety  standards,  that 
information  is  noted  on  the  same  form).  The  PTRS  form  is  used  to  document  all  activities  in  which 
inspectors  are  involved;  such  activities  include  accident/incident  investigation,  airman  certification,  flight 
school  certification,  etc.  Inspectors  also  use  paper-based  "job  aids",  which  are  essentially  checklists,  to 
assist  them  in  their  activities.  Filling  out  forms  and  following  checklists  are  the  types  of  activities  for 
which  the  NotePad  was  designed.  Therefore,  such  tasks  are  amenable  to  transfer  to  pen-computer 
technology.  Furthermore,  the  computer  allows  multiple  forms  to  be  linked  together  such  that  entries  in 
one  form  are  automatically  propagated  to  all  related  forms;  this  approach  would  eliminate  the  duplicate 
entry  of  data  which  currently  occurs.  Finally,  PTRS  forms  are  currently  recorded  in  paper  format  and 
given  to  data  entry  clerks  who  must  interpret  the  inspector’s  handwriting  and  transfer  the  data  to  the 
FSDO’s  local  computer-based  database  (which  feeds  into  the  national  PTRS  database).  The  PTRS  data 
collected  on  a  PPSS  will  be  in  a  format  that  can  be  directly  transferred  into  a  FSDO’s  local  PTRS 
database,  thus  eliminating  the  intermediate  manual  data  entry  step. 

Inspectors  must  also  have  access  to  large  amounts  of  information,  such  as  Federal  Aviation  Regulations, 
Inspector’s  Handbooks,  Airworthiness  Directives,  Advisory  Circulars,  etc.  Whereas  inspectors  must 
currently  retain  hard  copies  of  such  information  or  refer  to  the  FAA’s  mainframe  repository,  it  can  become 
cumbersome  to  access  and  track  this  information.  This  suggests  that  a  hypermedia  on-line  documentation 
system  can  be  beneficial  to  the  inspectors.  This  system  can  run  off  either  the  NotePad’s  internal  hard 
drive  or  an  external  CD  ROM  device.  Such  an  on-line  documentation  will  facilitate  rapid  access  to  up 
to  date  information. 

Based  on  initial  conversations  with  the  Flight  Standards  Service,  it  appears  that  a  PPSS  will: 

•  Provide  inspectors  with  an  integrated,  linked  form  system 

•  Provide  a  means  to  reduce  data  entry  performed  by  clerks 

•  Provide  on-line  documentation,  including  FARs.  handbooks,  etc. 


3.7.2  Phase  II:  Design  and  Development  of  a  Prototype  Portable  Performance  Support  System 

Goal:  Build  and  demonstrate  a  prototype  Portable  Performance  Support  System  to  support  Aviation 
Safety  Inspectors  doing  Ramp  Inspections. 

This  phase  will  involve  several  iterations  of  development  and  demonstration.  Rapid  prototyping  of  the 
PPSS  will  permit  us  to  demonstrate  the  sy.stcm  to  inspectors  for  quick  feedback  about  the  design  and 
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content  of  the  system.  We  will  work  closely  with  the  inspectors  during  this  phase  to  ensure  the  accuracy 
of  the  information  and  the  usability  of  the  PPSS.  While  these  evaluations  will  primarily  involve  the 
airworthiness  inspectors,  it  is  extremely  helpful  to  have  other  inspectors  evaluate  the  PPSS,  because  they 
can  provide  a  fresh  perspective  and  would  be  likely  to  identify  additional  areas  for  improvement. 

3.7J  Phase  III:  Create  a  Plan  for  the  Development  of  the  Prototype  System  into  a  Fully 
Operational  System 

Goal:  Create  a  plan  to  convert  the  prototype  PPSS  into  a  fully  operational  system  for  evaluation  and 
integration  into  the  work  environment. 

This  phase  will  require  a  formal  review  of  the  prototype  PPSS  to  identify  its  strengths  and  weakness. 
Following  this  review,  a  plan  will  be  developed  to  fully  implement  the  PPSS.  The  plan  will  include  the 
design  of  a  study  to  evaluate  the  effectiveness  of  the  PPSS  in  the  work  environment. 

3.8  SUMMARY 

Several  past  approaches  to  maintenance  job  aiding  were  discussed  with  respect  to  their  impact  on  human 
performance.  Such  approaches  have  typically  used  a  ‘machine  expert’  to  guide  technicians  through  the 
maintenance  process.  However,  the  ‘machine  expert’  paradigm  has  met  with  limited  success  in  operational 
environments  because  of  problems  with  unanticipated  variability  in  the  environment  (or  ‘brittleness’), 
extra-machine  knowledge,  and  inflexibility.  An  alternative  philosophy  to  developing  systems  was 
presented,  cooperative  systems,  in  which  both  the  human  and  the  computer  are  actively  involved  in  the 
problem  solving  process.  This  philosophy  advocates  a  change  in  perspective  toward  computers  as  tools 
to  assist  people  in  their  work,  rather  than  as  prostheses  to  overcome  human  ‘limitations’.  The  cooperative 
problem  solving  paradigm  capitalizes  on  the  strengths  of  humans  and  computers  in  order  to  improve  the 
performance  of  both.  A  study  which  compared  different  versions  of  a  Job  aiding  system  designed  with 
using  this  philosophy  was  presented,  along  with  implications  for  developing  maintenance  job  aids. 
Finally,  a  plan  for  developing  a  jxtrtable  performance  support  system  for  aviation  safety  inspectors  was 
presented. 
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Bogard,  W.  T.,  Palusamy,  S.  S.,  &  Ciaramitaro,  W.  (1988,  May).  Apply  automation  to  diagnostics, 
predictive  maintenance  in  plants.  Power,  pp.  27-32. 

The  ‘Advanced  Diagnostic  and  Predictive-Maintenance  System’  is  a  system  for  monitoring  and  diagnosing 
problems  at  nuclear  and  fossil  power  plants.  The  system  also  schedules  ‘predictive  maintenance’,  wherein 
maintenance  is  scheduled  based  on  performance  trends.  The  article  describes  system  modules,  with  an 
emphasis  on  trend  monitoring  and  preventive  maintenance. 

Bretz,  E.  A.  (1990,  July).  Expert  systems  enhance  decision-making  abilities  of  O&M  personnel. 
Electrical  World,  pp.  39-48. 

Overview  of  several  expert  systems.  Houston  Lighting  and  Power  Co.  uses  three  systems:  a  materials 
management  system  that  tracks  spare  parts  and  supplies,  a  maintenance  management  controls  system  that 
consolidates  and  standardizes  methods  for  requesting  and  tracking  maintenance,  and  an  expert  management 
scheduling  system  that  generates  reports  and  creates  schedules. 

Westinghouse  Electric  Corp.  developed  ‘Argus’,  an  alarm  response  advisor.  Argus  details  alarm  causes 
and  required  responses.  The  system  collects  data  on-line,  diagnoses  problems,  and  makes 
recommendations. 

Computational  Systems,  Inc.  developed  an  expert  system  for  vibrational  analysis  of  rotating  machinery. 

Chubu  Electric  Power  Co.  and  Toshiba  Corp.  developed  a  maintenance  support  expert  system  for  large 
turbine  generators.  The  system  handles  complex  and  time-con.suming  tasks.  A  engineer  enters  a  failure 
into  the  system  and  the  system  responds  with  other  damages  that  may  re.sult  from  a  suspected  root  cause, 
it  gives  standard  repair  methods  and  design  specifications,  and  it  displays  the  most  likely  failure  sources. 

Chubu  Elecbic  Power  Co.  and  Mitsubishi  Heavy  Industries,  Ltd.  developed  an  expert  system  for  boiler 
failure  analysis  and  maintenance  planning.  Failure  analysis  produces  the  most  probable  causes,  guidelines 
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for  inspection,  items  to  be  investigated,  repair  methods,  and  suggested  preventive  maintenance.  A 
maintenance  planning  subsystem  automatically  prepares  daily  repair  schedules,  work  estimation  plans,  and 
work  specifications. 

Byrd,  T.  A.,  Markiand,  R.  E.,  &  Karwan,  K.  R.  (1991,  July-August).  Keeping  the  helicopters 
flying-using  a  knowledge-based  tank  support  system  to  manage  maintenance.  Interfaces,  pp.  53-62. 

Discusses  a  knowledge-based  system  which  generates  reports  for  helicopter  maintenance.  The  system 
tracks  helicopters  and  notifies  maintenance  staff  of  which  helicopters  are  nearing  regular  inspections  or 
special  inspections.  Reports  specify  the  time-between-overhaul  components  that  will  require  maintenance 
soon  and  give  flying  schedules  prioritized  on  mission  and  maintenance  needs.  The  system  replaced  a 
cumbersome  manual  system. 

Callahan,  P.  H.  (1988,  January-February).  Expert  systems  for  AT&T  switched  network  maintenance. 
AT&T  Technical  Journal,  pp.  93-103. 

Describes  ‘Testing  Operations  Provisioning  Administration  System’  (TOPAS),  a  real-time,  distributed, 
multi-tasking  expert  system  for  switched  circuit  maintenance.  TOPAS  performs  trouble  analysis,  fault 
localization,  and  referral  for  network  switches.  TOPAS  is  claimed  to  do  maintenance  without  human 
intervention  or  consultation. 

Clancy,  C.  (1987,  November).  Qualitative  reasoning  in  electronic  fault  diagnosis.  Electrical 
Engineering,  pp.  141-145. 

Describes  an  expert  system  for  diagnosing  switch  mode  power  supplies  by  using  a  functional  model. 

Computer  oversees  maintenance.  (1992).  American  Water  Works  Association  Journal,  84,  107-108. 

Discusses  a  pc-based  preventive  maintenance  and  training  system  for  water  mains. 

Cue,  R.  W.  &  Muir,  D.  E.  (1991).  Engine  performance  monitoring  and  troubleshooting  techniques  for 
the  CF-18  aircraft.  Journal  of  Engineering  for  Gas  Turbines  and  Power,  1 13,  11-19. 

Discusses  the  In-flight  Engine  Condition  Monitoring  System  (lECMS)  as  a  foundation  for  ‘on-condition’ 
maintenance  of  fighter  aircraft  engines.  On-condition  maintenance  actions  are  undertaken  based  on  actual 
engine  conditions,  rather  than  as  preventive  maintenance.  lECMS  monitors  and  records  engine 
performance  parameters,  notifies  the  pilot  when  caution  should  be  exercised,  and  records  maintenance 
codes  when  an  operating  limit  has  been  exceeded.  Data  are  stored  on  a  removable  tape  cartridge.  The 
article  features  several  examples  of  data  indicating  normal  and  abnormal  operating  conditions. 

Culp,  C.  H.  (1989).  Expert  systems  in  preventive  maintenance  and  diagnostics.  ASHRAE  Journal,  31, 
24-27. 


Sales  article  on  using  expert  systems  for  heating,  ventilation,  and  air  conditioning  maintenance. 
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Dallimonti,  R.  (1987,  June  18).  Smarter  maintenance  with  expert  systems.  Plant  Engineering. 
pp.  51-56. 

An  introduction  to  the  prosp)ects  of  using  expert  systems  for  maintenance  Surveys  systems  from  Hughes 
Aircraft  Company,  Rockwell  International,  and  Campbell  Soup  Company. 

de  Kleer,  J.  (1990).  Using  crude  probability  estimates  to  guide  diagnosis.  Artificial  Intelligence.  45. 
381-391. 


Extension  of  ’General  Diagnostic  Engine’  discussion  in  de  Kleer  and  Williams. 

de  Kleer,  J.  &  Williams,  B.  C.  (1987).  Diagnosing  multiple  faults.  Artificial  Intelligence.  32. 
97-130. 


An  academic  discussion  of  a  ’General  Diagnostic  Engine’  for  diagnosing  multiple  faults.  Combines 
model-based  prediction  with  sequential  diagnosis  to  propose  measurements  to  diagnose  faults. 

Dobson,  R.,  &  Wild,  W.  (1989,  May).  Plant’s  computerized  maintenance  system  improves  operations. 
Power  Engineering,  pp.  30-32. 

A  Lotus  Symphony-based  system  automatically  processes  maintenance  clearances  for  power  plant. 

Dohner,  C.  V.,  &  Acierno,  S.  3.  (1989.  August).  Expert  systems  for  gas-turbine  powerplants 
passes  first  tests.  Power,  pp.  63-64. 

Citing  the  limited  amount  of  troubleshooting  and  diagno.stic  information  in  the  manufacturer’s  maintenance 
manuals,  EPRI  developed  an  expert  system  for  gas-turbine  power  plants.  A  portable  pc  uses  voice 
recognition  and  synthesis  and  links  to  a  pc  in  control  room.  The  control  room  pc  drives  a  video  disk 
player  and  a  printer. 

Doorley,  R.  (1988,  August).  Hydraulic  troubleshooting  using  an  expert  system.  Hydraulics  & 
Pneumatics,  pp.  91-92. 

Discusses  the  ‘MindMeld’  system  for  steel  mill  hydraulic  equipment  maintenance.  MindMeld  u.ses  test 
equipment  data  and  operator  information  to  determine  the  likely  cause  of  a  problem.  See  also  Doorley. 
(1989). 

Doorley,  R.  B.  (1989,  June  22).  Expert  systems  probe  h>draulic  faults.  Machine  Design,  pp. 
89-92. 

More  on  the  ‘MindMeld’  sy.stem  for  hydraulic  equipment  troubleshooting  in  steel  mills.  The  pc-bised 
sy.stem  focuses  on  faults  that  are  difficult  to  locate  and  which  require  extensive  dismantling  of  machineiy 
if  left  unrecognized. 
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Expert  system  from  AT&T  Bell  Laboratories  is  an  *ACE’  at  telephone  cable  analysis.  (1983. 
October).  Record,  p.  1. 

The  ‘Automated  Cable  Expertise’  system  identifies  trouble  spots  in  telephone  cable  systems.  ACE  gives 
repair  histories  of  problematic  areas  and  suggests  corrective  action.  The  system  is  automatic  and  runs 
daily. 

Expert  system  guides  tube-failure  investigations.  (1989,  August).  Power,  p.  85. 

Discusses  an  expert  system  for  boiler  tube  failure  diagnosis  and  corrective  action  (including  non¬ 
destructive  examination,  repair,  welding,  metallurgical  tests,  references).  The  system  an  be  used  to 
determine  tube  failure  mechanisms.  The  system  also  has  a  database  for  tube  history,  design,  inspection, 
maintenance  and  it  provides  context-sensitive  information  about  repair  practices.  PC-based,  linked  to  a 
slide  projector.  See  also  Smith.  (1989,  December). 

Expert  system  probes  beneath  the  surface.  (1990.  January).  Mechanical  Engineering,  p.  112. 

Britain’s  Central  Electricity  Generating  Board  developed  an  expert  system  for  monitoring  and  analyzing 
vibration  patterns  of  turbine  generators.  The  expert  system  uses  ‘deep  knowledge’  of  cause  and  effect 
relationships  in  turbine  generators.  The  goal  in  developing  the  system  was  to  transfer  initial  analysis  from 
specialist  staff  to  engineering/operations  staff. 

Expert  systems  to  hone  jet  engine  maintenance.  (1986,  April  21).  Design  News.  pp.  36-38. 

De.sctibes  an  expert  system  to  diagnose  engine  malfunctions  and  facilitate  preventive  maintenance  b> 
predicting  when  parts  must  be  replaced.  The  sy.stem  switches  maintenance  from  a  scheduled  replacement 
basis  to  an  ‘as-needed’  replacement  basis.  It  uses  qualitative  and  historical  maintenance  data.  The  pc- 
ba.sed  system  was  developed  by  the  General  Electric  R&D  Center  in  conjunction  with  GE’s  Aircraft 
Engine  Business  Group  for  the  Air  Force. 

FAA  and  NASA  design  program  to  improve  human  performance.  (1989,  May  29).  Aviaiion 
Week  &  Space  Technology,  p.  115. 

Outlines  Joint  FAA  and  NASA  effort  to  sfKntsor  human  factors  research. 

Foley,  \V.  L.  ,  &  Svinos,  J,  G.  (1989),  Expen  advisor  program  for  rod  pumping.  Journal  of 
Pelroleum  Technology.  41.  .'^94-4(K). 

‘EXPROD’  is  an  expen  adviser  program  for  rod-pumping  diagnostics  used  by  Chevron.  Die  progiam 
analyzes  field  data  to  identify  equipment  problems  and  recommend  solutions.  EXPROD  uses  .stati.stical 
pattern  recognition  in  conjunction  with  diagno.stic  rules.  .Some  worker  exp^-nise  is  still  required  to 
diagno.se  problems.  EXPROD  runs  on  a  microcomputer. 

Folley,  J.  D.,  &  Hritz.  R.  J.  (1987.  April).  Embedded  Al  expen  system  troubleshoots  autmiiated 
assembly.  Industrial  Engineering,  pp.  32-35. 
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Discusses  an  expert  system  to  assist  technicians  in  diagnosing  a  clutch  assembly  machine.  The  expert 
system  uses  a  computer-controlled  video  disk  to  indicate  what  the  technician  should  be  doing  or  looking 
at. 


Fault  lamps  above  stations  indicate  malfunctioning  assembly  stations.  The  technician  takes  a  monitor,  a 
two-button  control,  and  a  maintenance  cart  to  the  faulty  station  and  plugs  into  a  junction  box  connected 
to  the  computer  and  video  disk  player.  The  technician  selects  the  station  or  procedure  from  a  menu  and 
the  computer  specifies  tests  or  actions  with  graphics.  The  technician  supplies  data  and  the  computer 
specifies  the  next  action. 

GM  unveils  "Charley",  an  expert  machine  diagnostic  system.  (1988,  May).  I&CS,  pp.  4-7. 

Describes  vibration  analysis  expert  system  for  production  machinery  with  rotating  components.  Charley 
helps  mechanics:  1.  identify  parts  that  need  repair;  2.  repair  or  adjust  equipment  prior  to  failure;  3.  speed 
up  diagnosis;  4.  distribute  expertise;  and  5.  avoid  fixing  functioning  equipment.  See  also  Stovicek,  (1991). 

Gunhold,  R.,  &  Zettel,  J.  (1986).  System  12  in-factory  testing.  Electrical  Communication,  60(2),  \2S- 
134. 

Describes  a  diagnostic  expert  system  for  ITT  System  12  printed  circuit  board  assemblies. 

Hartenstein,  A.  (1988,  January).  Computer  system  controls  all  maintenance  activities.  Public 
Works,  p.  60. 

The  system  maintains  several  types  of  records  and  schedules  preventive  and  corrective  maintenance.  It 
also  issues  work  orders  and  monitors  progress.  It  is  a  database  system. 

Hill,  S.  (1990,  February).  Ask  the  expert.  Water  <6  Pollution  Control,  pp.  12-13. 

Concept  paper  that  discusses  possibilities  of  expert  systems  to  design  wastewater  treatment  facilities  and 
control  such  plants. 

Hughes,  D.  (1988,  March  7).  Digital  develops  special  applications  to  meet  diverse  aerospace 
needs.  Aviation  Week  &  Space  Technology,  pp.  51-53. 

Jet  fighter  uses  AI  as  troubleshooter.  (1986,  July  21).  Design  News,  p.  20. 

More  on  AIMES.  See  also  "McDonnell  Douglas..." 

Keller,  B.  C.  &  Knutilla,  T.  R,  (1990,  September).  U.S.  Army  builds  an  AI  diagnostic  expert 
system,  by  soldiers  for  soldiers.  Industrial  Engineering,  pp.  38-41. 

Describes  the  ‘Pulse  Radar  Intelligent  Diagnostic  Environment’  for  troubleshooting  the  l\ilse  Acquisition 
Radar  of  a  Hawk  missile  system. 
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King,  I.  J.,  Chianese,  R.  B.,  &  Chow,  M.  P.  (1988,  December).  Plant  diagnosUcs  relies  on  AI 
transmissions  from  remote  site.  Power,  pp.  57-60. 

Describes  a  suite  of  on-line  power  plant  diagnostic  systems  developed  by  Westinghouse  Electric  Corp. 
Development  goals  were  to  maximize  availability  and  efficiency  and  reduce  forced-outage  rates  of  turbine 
generators.  The  systems  identify  worn  or  damaged  components  early.  ‘ChemAID’  diagnoses  problems 
in  the  steam/water  cycle;  it  determines  the  type,  severity,  and  location  of  water  chemistry  problems. 
ChemAID  assists  the  operator  in  determining  the  need  for  immediate  or  delayed  action.  It  can  also  serve 
as  a  consultant. 

‘TurbinAID’  diagnoses  problems  in  steam  turbines.  It  diagnoses  the  condition  and  thermodynamic 
performance  of  turbines  and  reports  current  and  target  performance  parameters. 

‘GenAID’  monitors  trends  for  gas-cooled  generators. 

Kinnucan,  P.  (1985,  November).  A  maintenance  expert  that  never  sleeps.  High  Technology, 
pp.  48-9. 


The  ‘Intelligent  Machine  Prognosticator’  (IMP)  is  an  expert  system  for  maintenance  of  an  epitaxial  reactor 
(equipment  that  ‘grows’  additional  silicon  crystals  on  silicon  wafers).  IMP  diagnoses  faults  and 
recommends  repair  procedures.  The  system  was  developed  because  vendor  supptort  was  difficult  to  obtain. 
IMP  reportedly  reduced  repair  time  by  36%. 

Kolcum,  E.  H.  (1989,  January  2).  Growing  flight,  maintenance  simulator  market  attracts  many 
competitors.  Aviation  Week  &  Space  Technology,  pp.  91-93. 

Discusses  proposed  military  exp)enditures  on  simulators  for  maintenance  training. 

Layton,  C.F.  (1992).  An  investigation  of  the  effects  of  cognitive  tools  on  human  adaptive  planning  in 
the  domain  of  enroute  flight  planning.  Doctoral  dissertation.  The  Ohio  State  University,  Columbus, 
OH. 

Discusses  a  study  which  compared  three  different  levels  of  computer  support  for  enroute  flight  planning. 
The  study  compared  the  behaviors  of  thirty  professional  airline  pilots  assigned  randomly  to  each  of  three 
treatment  conditions  (ten  subjects  per  condition,  each  condition  consisted  of  a  different  form  of  computer 
support).  The  subjects  were  trained  on  system  use  prior  to  solving  four  enroute  flight  planning  scenarios. 
The  focus  of  the  research  was  not  on  the  principles  used  to  design  the  enroute  flight  planning  systems, 
rather  it  was  on  what  characteristics  the  designs  shared  and  what  characteristics  were  unique  to  a  particular 
system.  The  purpose  of  the  research  was  to  study  how  these  three  system  designs,  as  examples  of  broader 
classes  of  planning  assistance  tools,  affected  enroute  flight  planning,  as  an  example  of  adaptive  planning. 
The  goals  of  this  study  were  to  develop  a  better  understanding  of  the  adaptive  planning  prtK'Css  and  to 
develop  recommendations  for  designing  tools  to  support  that  process. 

Maintenance  expert  in  a  briefcase.  (1986,  April).  High  Technology,  p.  9. 
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‘Mentor’  is  a  portable  expert  system  for  routine  maintenance  and  diagnosis  of  air  conditioners.  Mentor 
keeps  service  records  of  each  piece  of  equipment.  The  emphasis  is  on  preventive  maintenance. 

Majstorovic,  V.  D.  (1990,  October).  Expert  systems  for  diagnosis  and  maintenance;  State  of 
the  art.  Computers  in  Industry,  p.  43-68. 

Discussion  of  typical  expert  system  components  and  survey  of  diagnosis  and  maintenance  expert  systems. 
Discusses  expert  systems  for  maintenance  of  flexible  manufacturing  systems. 

McDonnell  Douglas  flight  tests  AI  maintenance  data  processor.  (1986,  February  17).  Aviation  Week 
&  Space  Technology,  p.  69. 

Describes  ‘Avionics  Integrated  Maintenance  Expert  System’  (AIMES)  forF/A-18.  AIMES  gathers  aircraft 
data  and  creates  flight  files  for  later  analysis.  Production  rules  detect  and  isolate  avionic  failures  at  the 
electronic  card  level.  Analysis  provides  fault  data,  the  card  name,  and  the  reasoning  that  led  to  the  fault 
isolation  conclusion.  AIMES  is  a  self-contained  on  board  system  with  a  microprocessor  and  a  data  storage 
cassette.  AIMES  includes  BITE.  See  also  "AI  to  help  keep..." 

McDowell,  J.  K.,  &  Davis,  J.  F.  (1991).  Managing  qualitative  simulation  in  knowledge-based 
chemical  diagnosis.  AICliE  Journal,  37,  569-580. 

An  academic  discussion  of  an  approach  for  dealing  with  multiple  interacting  faults  with  an  expert  system. 

Melhem,  H.  G.,  &  Wentworth,  J.  A.  (1990,  March).  FASTBRID:  An  expert  system  for  bridge  fatigue. 
Public  Roads,  pp.  109-117. 

‘Fatigue  Assessment  of  Steel  Bridges’  is  a  training  aid  for  bridge  inspection  and  planning  remedial  actions. 
It  is  also  an  advisory  system  for  evaluation  of  bridges.  The  advisory  system  helps  organize  fatigue 
inspection,  evaluate  inspection  results,  and  determine  a  course  of  action. 

Miller,  D.  M.,  Mellichamp,  J.  M.,  &  Wang,  J.  (1990.  November).  An  image  enhanced,  knowledge 
based  expert  system  for  maintenance  trouble  shooting.  Computers  in  Industry,  pp.  187-202. 

Describes  an  expert  system  for  diagnosis  of  the  electrical/hydraulic  system  of  an  clechic  utility  vehicle. 
The  vehicle  system  is  difficult  to  diagnose  because  faults  can  be  masked  by  behavior  of  equipment.  The 
expert  system  displays  a  limited  number  of  photographs  of  a  vehicle.  The  expert  system  is  pc  based. 

Miller,  F.  D.,  Rowland,  J.  R.,  &  Siegfried,  E.  M.  (1986,  January).  ACE:  An  expert  system 
for  preventive  maintenance  operations.  Record,  pp.  20-25. 

Discusses  the  ‘Automated  Cable  Expertise’  (ACE)  system  for  telephone  cable  analysis.  ACE  is  an 
automatic  report  generator  which  runs  daily.  Sell  also  "Expert  system  from  AT&T  Bell  Laboratories...". 

Moiadian,  S.,  Thompson,  E.  D.,  &  Jenkins,  M.  A.  (1991,  May).  New  idea  in  on-line  diagnostics 
improves  plant  performance.  Power,  pp.  49-51. 


Emerging  Technologies  for  Maintenance  Job  Aids 


Discusses  Westinghouse  expert  systems,  particularly  GenAlD.  See  also  King,  Chianese,  and  Chow  (1988, 
December). 

Nelson,  B.  C,,  &  Smith,  T.  J.  (1990).  User  interaction  with  maintenance  information:  A  performance 
analysis  of  hypertext  versus  hardcopy  formats.  Proceedings  of  the  Human  Factors  Scxiety  34th  Annual 
Meeting,  229-233. 

Research  paper  which  discusses  experimental  results  of  comparing  hypertext  versions  of  maintenance 
manuals  with  hardcopy  versions  of  those  manuals.  Subjects  were  slower  with  hypertext  than  hardcopy, 
but  preferred  hypertext.  Enhanced  versions  of  manuals  (hypertext  and  hardcopy)  improved  access  to 
information  and  comprehension  of  that  information,  but  subjects  did  not  recognize  improvements. 

Nordwall,  B.  D.  (1989,  June  19).  CTA  develops  new  computer  system  to  speed  civil  aircraft 
maintenance.  Aviation  Week  &  Space  Technology,  pp.  153-157. 

Describes  the  ‘Automated  Maintenance  Management  System’  (AMMS)  and  the  ‘Mobile  Enhanced 
Comprehensive  Asset  Management  System’  (MECAMS)  developed  by  CTA.  Inc.  The  AMMS  collects 
inflight  information  on  F/A-18  aircraft  engines  and  stores  it  on  a  floppy  disk  for  later  analysis  by 
MECAMS.  MECAMS  runs  on  a  laptop  computer  connected  to  a  minicomputer,  which  in  turn  could  be 
connected  via  satellite  or  phone  link  to  a  database  of  troubleshooting  and  logistics  information.  MECAMS 
first  identifies  periods  when  engine  performance  parameters  are  exceeded,  then  it  assists  technicians  in 
troubleshooting  and  maintaining  the  engines.  CTA  indicated  that  the  F/A-18  engine  system  was  a  proof 
of  concept  and  was  proposing  that  the  same  system  could  be  extended  to  civil  aircraft  engines  and 
avionics. 

NYNEX  cuts  costs  in  40  offices  using  expert  system.  (1990,  September).  Industrial  Engineering,  pp. 
81-82. 


‘Maintenance  Administrator  Expert’  (MAX)  diagno.ses  problems  with  residential  and  small  business 
telephone  service.  MAX  interprets  trouble  report  data  in  5-10  sec.  as  opposed  to  the  5-10  min.  a  human 
requires.  MAX  dispatches  the  correct  technician  and  reduces  false  dispatches.  The  system  reportedly 
saves  $4-6  million/yr.  MAX  was  developed  by  NYNEX  Science  and  Technology  Center  and  runs  on  a 
Sun  3/260. 

Paula,  G.  (1990,  March).  Expert  system  diagnoses  transmi.ssion-line  fault.s.  Electrical  World. 
pp.  S41-S42. 

Describes  an  automatic  expert  system  that  uses  a  mathematical  model  of  a  power  tran.smi.s.sion  system  for 
fault  diagnosis.  The  expert  system  activates  upon  failure  of  the  transmission  system  and  produces  a 
prioritized  list  of  possible  fault  locations  and  their  corre.sponding  failure  probabilities. 

Ray,  A.  K.  (1991,  June).  Equipment  fault  diagnosis-a  neural  network  approach.  Computers 
in  Industry,  pp.  169-177. 

Describes  a  system  designed  for  mechanical  equipment  diagnosis  in  the  steel  industry. 
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Reason,  J.  (1987,  March).  Expert  systems  promise  to  cut  critical  machine  downtime.  Power, 
pp.  17-24. 

Discusses  prospects  for  continuous  vibration  monitoring  systems,  leading  to  automatic  diagnostic  systems. 
The  article  also  highlights  several  expert  systems  in  the  power  industry.  ‘Turbomac’  is  an  expert  system 
for  diagnosing  vibrations  in  large  turbo  machinery,  particularly  power  generating  facilities. 

‘GenAid’  is  an  expert  system  developed  by  Westinghouse  Electric  Corp.  for  diagnosing  hydrogen-cooled 
electric  generators.  The  purpose  of  GenAid  was  to  avoid  catastrophic  failure.  See  also  King,  Chianese, 
and  Chow  (1988,  December). 

The  Central-Hudson  Electric  &  Gas  Corp.  developed  an  expert  system  for  scheduling  outages.  The 
purpose  of  the  system  was  to  reduce  the  number  of  scheduled  outages  without  compromising  equipment 
integrity.  The  system  schedules  outages  for  preventive  maintenance  at  the  first  sign  of  trouble. 

‘Transformer  Oil  Gas  Analyst’  is  an  expert  system  for  detecting  and  diagnosing  signs  of  impending 
transformer  failure. 

General  Electric  developed  an  expert  system  for  turbines.  The  system  is  portable,  links  to  a  video  disk 
display,  and  uses  voice  recognition  for  form  fill-in  or  multiple  choice  input. 

Rodriguez,  G.,  &  River,  P.  (1986,  July).  A  practical  approach  to  expert  systems  for  safety  and 
diagnostics.  InTech,  pp.  53-57. 

Describes  an  expert  system  for  diagnosis  of  a  400/200  KV  hybrid  gas  insulated  substation  of  the  Laguna 
Verde  Nuclear  Power  Station  in  Mexico.  Engineers  and  literature  provided  information  to  build  fault  trees 
to  model  loss  of  current  to  safety-related  control  boards.  System  objectives  were  the  timely  diagnosis  of 
abnormal  events  or  transients,  and  analysis  of  events  leading  to,  and  consequences  of,  an  abnormal 
situation. 

Roth,  E.  M.,  Bennett,  K.  B.,  &  Woods,  D.  D.  (1987).  Human  interaction  with  an  "intelligent" 
machine.  International  Journal  of  Man-Machine  Studies,  27.  479-525. 

Reports  a  study  investigating  technicians  using  an  expert  system  to  troubleshoot  an  electro-mechanical 
device.  The  article  documents  common  problems  of  ‘machine  expert’  problem  solving  systems.  Only 
technicians  who  were  actively  involved  in  the  problem  solving  prcx:ess  and  who  performed  actions  in 
addition  to  those  requested  by  the  expert  sy.stem  successfully  completed  the  sample  tasks.  Technicians 
who  responded  passively  to  expert  system  requests  were  unable  to  solve  the  problems.  This  study  should 
be  read  by  all  those  interested  in  improving  human  performance  through  computational  support. 

Rowan,  D.  A.  (1988,  May).  AI  enhances  on-line  fault  diagnosis.  InTech,  pp.  52-55. 

Describes  ‘Fault  Analysis  Consultant’  (Falcon)  for  on-line  fault  diagnosis  in  a  commercial  chemical  plant. 
Falcon  reasons  from  first  principles  and  heuristic  knowledge.  Falcon  went  on-line  in  1988. 
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Rustace,  P.  (1988,  June  9).  Knowledge  of  an  expert  compressed  on  computer.  The  Engineer, 
p.  44. 

Discusses  monitoring  expert  system  for  gas  turbine-driven  compressor  sets. 

Save  plant  know-how  with  expert  systems.  (1987,  August).  Electrical  World,  pp.  54-55. 

Discusses  expert  system  to  resolve  power  plant  control  room  alarms. 

Schaaf,  J.  R.  (1985,  September).  Computerization  of  sewer  maintenance  scheduling.  Public 
Works,  pp.  128-129. 

‘Computerization  of  Sewer  Maintenance  Operations’  (COSMO)  schedules  routine  cleaning  operations. 
COSMO  also  tracks  performance,  debris  severity,  and  maintenance  history  and  uses  this  information  to 
set  cleaning  priorities  and  schedule  sewer  cleaning.  Database  system. 

Shifrin,  C.  A.  (1985,  October  28).  Eastern  computer  system  reduces  maintenance  layovers,  staff 
levels.  Aviation  Week  &  Space  Technology,  pp.  40-45. 

Describes  a  computer  system  that  includes  computerized  work  cards  to  be  filled  in  by  maintenance 
personnel.  The  system  will  also  produce  hard  copies  of  tasks  and  checks.  The  work  cards  contain 
detailed  instructions,  warnings,  and  notes.  The  system  aids  in  capacity  planning  through  tracking  line 
slippage,  schedule  constraints,  and  manpower  limitations. 

The  article  also  discusses  a  parts  tracking  and  scheduling  system. 

Smith,  D.  J.  (1987,  May).  Diagnostic  analysis  leads  the  way  in  preventive  maintenance.  Power 
Engineering,  pp.  12-19. 

Describes  ‘Process  Diagnosis  System’  (PDS)  developed  by  Weslinghouse  R&D  Center  and  Carnegie 
Mellon  University.  PDS  diagnoses  problems  with  steam  generators  and  provides  recommendations  and 
procedures  for  fixing  the  problems.  PDS  is  claimed  to  cut  down  on  ‘over-maintenance’,  but  prevent 
‘breakdown  maintenance’.  The  system  monitors  the  condition  of  an  operational  plant  and  analyzes  plant 
data  to  detect  incipient  faults  and  deterioration.  PDS  uses  this  information  to  diagnose  faults  and  predict 
the  duration  of  safe  operation  without  maintenance.  PDS  also  recommends  preventive  maintenance  tasks 
to  be  performed  during  scheduled  down  periods. 

Smith,  D.  J.  (1989,  January).  Artificial  intelligence-today’s  new  design  and  diagnostic  tool. 
Power  Engineering,  pp.  26-30. 

Overview  of  AI  applications  in  maintenance.  The  article  de.scribes  the  EPRl-developed  ‘Gas  Turbine 
Expert  Sy.stem’  and  a  Westinghouse-developed  system  for  on-line  valve  diagnosis 

Smith,  D.  J.  (1989,  December).  Intelligent  computer  systems  enhance  power  plant  operations. 
Power  Engineering,  pp.  2 1  -26. 
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Discusses  several  expert  systems  in  use  in  the  power  industry.  The  ‘ESCARTA’  system  for  reducing 
boiler  tube  failures  has  several  uses:  it  permits  an  engineer  to  track  down  a  failure  mechanism,  it  provides 
non-destructive  testing  procedures,  it  provides  welding  procedures,  it  provides  corrective  actions  for  failure 
repair,  and  it  facilitates  training.  ESCARTA  will  show  operators  the  correct  procedures  for  investigating 
tube  failures.  It  also  suggests  root  causes  that  conld  have  led  to  a  failure.  ESCARTA  is  pc  based. 

‘Coal  Quality  Advisor’  assesses  coal  quality.  The  system  helps  assess  cost  and  performance  aspects  of 
using  different  coals  or  coal  blends.  It  is  pc  based. 

‘Smart  Operator’s  Aid  for  Power  Plant  Optimization’  diagnoses  causes  of  heat  rate  degradation  on  oil-  and 
gas-fired  power  plants.  The  system  justifies  its  diagnosis  through  logic  trees  or  messages.  It  also 
recommends  corrective  actions. 

‘TurbinAID’,  ‘GenAID’,  and  ‘ChemAID’  make  up  a  suite  of  expert  systems  for  diagnosis  of  turbine 
generators.  The  systems  were  developed  by  Westinghouse  Electric  Corp.  See  also  King.  Chianese.  and 
Chow  (1988,  December). 

Stacklin,  C.  A.  (1990,  June).  Pairing  on-line  diagnostics  with  real-time  expert  systems.  Power. 
pp.  55-58. 

Concept  paper  on  using  expert  systems  to  reduce  un.scheduled  down  time.  Discusses  fault  trees,  failure 
modes  and  effects  analyses,  and  pattern  recognition. 

Stein,  K.  J.  (1988,  March  14).  Expert  system  technology  spurs  advances  in  training,  maintenance. 
Aviation  Week  &  Space  Technology,  pp.  229-2.13. 

Discusses  emerging  expert  system  technology  in  maintenance.  The  Navy  Sea  System  Command’s 
Integrated  Diagnostic  Support  System  (IDSS)  collects  fault-related  data  and  i.solates  faults.  The  system 
continually  builds  its  knowledge  base  so  that  it  becomes  more  efficient  with  use.  IDSS  will  i.solate  faults 
to  the  microchip  level  by  using  fault  trees.  IDSS  uses  a  touch  screen,  a  flat  panel  display,  and  an 
interactive  maintenance  tutorial  on  a  video  disk.  IDSS  is  expected  to  aid  systems  designers  in  building 
self-diagnostics  in  new  avionics  systems. 

The  article  also  describes  Rex-MATE  for  use  with  the  USAF  modular  automatic  test  equipment. 
Stovicek,  D,  (1991,  February).  Cloning  knowledge.  Automalion,  pp.  46-48. 

Discusses  General  Motors’  ‘Charley’  system  for  vibration  analysis  of  production  equipment.  Charley  is 
an  expert  system  developed  by  GM’s  Advanced  Engineering  Staff  and  is  named  and  modeled  after  a 
retired  vibration  analysis  expert.  Charley  contains  three  modules:  1.  a  rule-ba,se  for  vibration  analysis, 
2.  a  ‘vibration  signature’  databa.se.  which  contains  the  vibration  curves  of  the  various  pieces  of  equipment, 
and,  3.  a  machine  databa.se,  which  contains  hi.storical  data  on  each  machine.  Charley  is  used  for  failure 
diagnosis,  preventive  maintenance,  and  training.  Charley  can  be  u.sed  to  answer  ‘what  if  questions  and 
explains  diagnosis  strategies.  Runs  on  a  Sun  computer.  Sec  also  "GM  unveils..." 

Suchman,  L.  A.  (1987).  Plans  and  situated  actions:  The  problem  of  human  machine  communication. 
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Discusses  human  action  with  respect  to  circumstantial  variability  and  the  difficulties  in  communicating 
such  variability  to  a  machine.  Rigid  problem  solving  on  the  part  of  the  machine  and  misinterpretations 
on  the  part  of  the  human  are  some  of  the  obstacles  to  successful  human  computer  interaction  discussed. 

Sutton,  G.  (1986,  January).  Computers  join  the  maintenance  team.  WATER  Engineering  & 
Management,  pp.  31-33. 

‘Maintenance  Management  System’  (MMS)  for  water  and  wastewater  treamient  facilities.  MMS  tracks 
organization  performance,  determines  resource  utilization  and  work  backlog,  and  makes  personnel  and 
resource  utilization  projections.  It  also  schedules  preventive  maintenance.  MMS  is  a  database  system. 

Thandasseri,  M.  (1986).  Expert  systems  application  for  TXE4A  exchanges.  Electrical  Communication, 
60(2),  154-161. 

Describes  ‘Advance  Maintenance  Facility’,  an  expert  system  for  fault  identification.  Normally  the  system 
controls  interactions,  but  it  can  be  ‘controlled’  by  an  operator.  Output  is  corrective  action  and  post-repair 
tests. 

Toms,  M.,  &  Patrick,  J.  (1987).  Some  components  of  fault-finding.  Human  Factors,  29(5), 
587-597. 

Research  paper  on  human  performance  in  network  fault-finding  tasks. 

Turpin,  B.  (1986,  March  3).  Artificial  intelligence:  Project  needs.  Df.jtgn  At'vv^,  pp.  104-1 14. 

Discusses  expert  system  developed  by  Campbell  Soup  for  diagnosing  problems  with  ‘cookers’.  The 
system  was  built  to  replace  a  retiring  technician  with  25  years  of  experience. 

The  article  also  discusses  an  expert  system  (Intelligent  Machine  I^ognosticaior)  developed  by  Texas 
Instruments  for  epitaxial  reactor  maintenance.  See  also  Kinnucan  (1985,  November). 

Users  get  expert  advice  on-site.  (1987,  March  12).  ENR,  p.  21. 

The  ‘Exstra’  expert  system  troubleshoots  mechanical  equipment  failures  in  compressors,  water  pumps,  and 
other  rotating  equipment.  Exstra  lists  possible  conclu.sions  with  likelihcxid  ratings.  It  afso  explains  why 
it  asks  particular  questions.  Exstra  is  VAX  ba.sed. 

Uttley,  A.  (1985,  October  17).  Computer  ‘expert’  helps  find  faults.  The  Engineer,  p.  76. 

Describes  expert  system  shell  software. 

Vanpelt,  H.  E.,  &  Ashe,  K.  U.  (1989,  April).  Radiation  exposure  reduced  with  computer-aided 
maintenance.  Power  Engineering,  pp.  40-42. 
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Describes  ‘Plant  Radiological  Status’  (PRS),  a  three  dimensional  computer  model  of  a  power  generating 
station  and  its  equipment,  developed  by  Duke  Power  Co.  PRS  is  claimed  to  reduce  maintenance  time  and 
radiation  exposure  by  supporting  planning  activities.  PRS  identifies  maintenance  interference  problems 
(eg.  restricted  access)  and  available  work  space.  PRS  aiso  facilitates  access  and  interpretation  of 
radiological  conditions  in  the  plant;  it  identifies  hot  spots  and  contaminated  areas. 

Woods,  D.  D.  &  Roth,  E.  M.  (1988).  Cognitive  engineering:  Human  problem  solving  with 
tools.  Human  Factors,  30(4),  415-430. 

Concept  paper  which  describes  the  fundamental  aspects  of  cognitive  engineering.  According  to  the 
authors,  "Cognitive  engineering  is  an  applied  cognitive  science  that  draws  on  the  knowledge  and 
techniques  of  cognitive  psychology  and  related  disciplines  to  provide  the  foundation  for  principle-driven 
design  of  person-machine  systems."  (p.  415  ).  Like  Roth,  Bennett,  &  Woods  (1987),  this  article  should 
be  read  by  all  those  interested  in  supporting  human  performance  through  computational  support. 

Yu,  C'C.,  &  Lee,  C.  (1991).  Fault  diagnosis  based  on  qualitative/quantitative  process  knowledge. 
AIChE  Journal,  37,  pp.  617-628. 

An  academic  discussion  of  an  approach  for  combining  qualitative  and  quantitative  reasoning  in  an  expert 
system  through  fuzzy  logic.  Aside;  Although  fuzzy  logic  is  a  technique  that  is  often  used  in  artificial 
intelligence  because  it  doesn’t  carry  the  overhead  associated  with  Bayesian  or  other  probability  theory 
based  methods,  it  also  lacks  the  mathematical  rigor  of  the  latter  methods. 
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The  FAA  Aviation  Maintenance  Human  Factors  Hypermedia  System 
4.0  INTRODUCTION 

The  Federal  Aviation  Administration  (FAA)  Aviation  Maintenance  Hypermedia  Information  System  (HIS) 
is  part  of  the  Office  of  Aviation  Medicine  (AAM)  Human  Factors  in  Aviation  Maintenance  Research 
(HFAMR)  program.  The  goal  of  the  HIS  project  is  to  create  new  tools  and  methods  for  information 
access  and  use  and  to  provide  these  tools  and  methods  for  support  of  other  HFAMR  activities  (e.g., 
training  and  Job  aiding  systems).  These  tools  and  methods  provide  the  vital  information  access  component 
for  any  computer-based  aviation  maintenance  integrated  information  system  (Johnson  and  Norton,  1992). 

In  its  present  state  the  HIS  provides  an  environment  to  create  and  explore  large  collections  of  related 
information.  The  HIS  provides  a  simple,  yet  powerful,  way  of  creating  and  following  associations 
between  related  pieces  of  information.  Using  the  HIS,  the  user  can  browse  and  view  information  in  a 
variety  of  ways.  This  flexible  method  of  information  access  and  utilization  is  not  available  to  users  of 
conventional  text  retrieval  systems. 

Publications  and  presentations  from  the  HFAMR  program  are  being  placed  into  the  HIS  for  the  initial 
domain.  This  material  includes  presentations  from  the  first  five  Human  Factors  in  Aviation  Maintenance 
Conferences,  as  well  as  complete  material  from  the  HFAMR  Phase  1  (Shepherd,  et  al.,  1991)  and  Phase 
II  Progress  Reports.  The  result  is  an  on-line  document  that  employs  the  latest  hypermedia  software 
technology.  More  importantly,  software  tools  and  mcthtxls  have  been  con.structed  that  provide  the  ability 
to  quickly  store,  locate,  and  deliver  information  for  a  variety  of  aviation  maintenance  tasks. 

4.1  PRACTICAL  ASPECTS  OF  HYPERMEDIA 

The  fundamental  nature  of  computer-based  hypermedia  is  to  structure  information  in  a  fashion  that  can 
be  quickly  and  randomly  located.  Conventional  forms  of  media  delivery  (e.g.,  books,  television,  video, 
audio  tapes,  etc.)  tend  to  be  linear  in  nature.  The  reader  of  this  linear  information  typically  starts  al  the 
beginning  of  a  presentation  and  progresses  along  a  predetermined  path  (e.g.,  turning  to  the  next  page  in 
a  book,  or  being  forced  to  sit  through  commercials  while  watching  TV).  The  reader  or  viewer  of  this 
information  often  has  little  choice  in  determining  what  information  comes  next. 

Hypermedia,  on  the  other  hand,  arranges  information  in  a  form  in  which  a  reader  or  viewer  can  "bounce" 
around  between  different  segments  of  information  (similar  to  a  reader  of  a  mystery  book  flipping  to  the 
end  of  the  story;  or  a  reader  of  a  technical  manual  first  looking  up  a  term  in  a  index,  and  then  going  to 
the  correct  page  in  the  manual).  This  idea  of  associations,  or  links,  between  segments  of  infe^rmation  is 
a  core  feature  of  hypermedia  systems.  An  example  of  a  typical  hypermedia  system  is  illustrated  in  Figure 
4.1. 

For  example,  a  hypermedia  version  of  a  car  maintenance  manual  would  first  present  the  reader  with  a 
diagram  of  a  car.  From  this  diagram,  the  reader  would  point  to  a  particular  portion  of  the  car  (e.g.,  the 
engine)  and  request  more  information.  The  hypermedia  system  would  then  display  a  diagram  of  the 
engine.  The  reader  would  point  to  an  engine  component  (e.g.,  the  battery)  and  the  hypermedia  system 
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would  then  present  a  verbal  description  of  the  battery  and  list  possible  troubleshooting  advice  dealing  with 
common  battery  problems. 

For  the  purposes  of  this  paper,  the  term 
"hypermedia"  is  used  because  the 
research  is  not  restricting  the  usage  of 
this  system  to  only  text  -  "hypertext." 

The  scope  of  this  system  has  been 
extended  to  include  text,  still  images, 
graphical  animation,  audio,  and  video  - 
"hypermedia". 

One  of  the  advantages  of  using 
hypermedia  technology  is  its  strong 
support  for  e?sy  access  to  items  stored 
in  large  colleciions  of  information. 

Typically,  structures  of  large 
collections  of  information  are  complex. 

This  leads  to  various  indexing  schemes 
that  are  used  to  aid  readers  in  locating 
information  (e.g.,  tables  of  contents 
and  indices  for  books,  Dewey  Decimal 
and  Library  of  Congress  indexing  for 
libraries).  Even  with  these  schemes 
however,  someone  searching  for  a 
particular  piece  of  information  may 
still  have  limited  success  in  locating 
the  desired  information.  Hypermedia  4.1  A  Typical  Hypermedia  System 

systems  extend  the  indexing  of 

information  by  providing  associations,  or  links,  between  particular  related  pieces  of  information.  The  idea 
is  to  provide  readers  with  the  ability  to  access  a  general  location  in  the  information  base  using  various 
indexes  and  searching  schemes.  Then  the  reader  can  browse  the  general  locations  using  specific 
associations  to  locate  the  desired  information.  To  support  browsing  and  exploration  of  information,  the 
presence  of  anchors  (also  often  called  buttons)  is  employed  to  signal  the  reader  that  related  information 
is  available  (Duchaslel,  1990). 

The  Intermedia  project  at  Brown  University  is  a  very  good  example  of  hypermedia  technology  being 
implemented  for  a  large  information  space.  Over  a  thousand  pieces,  or  "nodes,"  of  information  are 
connected  by  over  two  thousand  links.  Intermedia  pre.scnts  the  user  with  a  graphical  information  brow.ser, 
a  set  of  graphical  editors  for  text,  graphics,  limelines,  animations,  and  videodisc  data,  a  browser  for  link 
information,  and  the  ability  to  create  and  traverse  links  between  any  two  selections  in  any  document  in 
the  sy.stem  (Haan,  Kahn,  Riley,  Coombs,  and  Meyrowit/.,  1992).  Tlie  Intermedia  system  illustrates  the 
ease  in  which  information  can  be  placed  into  hypermedia  systems  and  how  that  information  can 
immediately  benefit  other  users  of  the  system. 
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The  Artifact-Based  Collaboration  (ABC)  project  at  the  University  of  North  Carolina  (Smith  and  Smith. 
1991)  is  a  good  example  of  hypermedia  technology  being  used  to  support  groups  of  individuals  working 
together  to  build  large,  complex  structures.  ABC  has  five  comptments  that  include  a  graph  server,  a  set 
of  graph  browsers,  a  set  of  data  application  programs,  a  shared  window  conferencing  facility,  and  real-time 
video  and  audio.  ABC  also  demonstrates  the  ability  of  hypermedia  technology  to  assimilate  new 
information  and  to  quickly  disseminate  the  information  for  immediate  use. 

4.2  THE  FAA  HYPERMEDIA  SYSTEM 

The  overall  goal  of  the  FAA  Hypermedia  Information  System  (HIS)  development  is  to  study  the  prospects 
and  problems  of  creating  an  electronic  document.  The  project  will  also  determine  how  these  technologies 
can  be  used  to  improve  the  delivery  of  information  to  support  aviation  maintenance.  Although  focused 
on  one  particular  discipline,  this  development  sho  jld  produce  results  that  help  to  guide  the  development 
of  many  kinds  of  future  aviation  maintenance  support  applications,  hereafter  referred  to  as  FAA  integrated 
information  systems. 

4.2.1  Research  Phases 

The  development  of  the  HIS  is  divided  into  two  phases  that  will  be  conducted  over  a  two  year  period. 
The  work  began  in  July  of  1991.  The  first  eight  months  (Phase  1)  were  dedicated  to  the  prototype  design 
and  development.  The  results  of  this  phase  have  been: 

1)  A  functional  on-line  version  of  the  Third  FAA  meeting  on  Human  Factors  Is.sues  in  Aircraft 
Maintenance  and  Inspection.  This  version  is  functionally  equivalent  to  the  final  version  to  be 
implemented  in  1992. 

2)  Specifications  to  authors  of  future  FAA  HF  conference  papers  to  aid  the  incorporation  of  new 
information  into  the  HIS. 

3)  Reusable  technology  base  for  indexing  and  retrieving  hypermedia  text  and  graphics. 

The  second  phase  is  currently  under  development  and  is  divided  into  four  tasks.  The  first  task  focuses  on 
incorporating  the  remaining  four  conference  proceedings  into  the  HIS  and  includes  the  production  of  a 
CD-ROM  version  of  the  HIS.  The  second  task  focu.ses  on  enhancement  of  the  Hypermedia  technology 
base.  The  third  task  will  involve  suppxirt  for  the  transition  of  Hypertext  technology  to  FAA  integrated 
information  system  research.  The  fourth  task  will  focus  on  demonstrating  and  reporting  research  results 
to  fellow  HF  team  members,  the  project  sponsor,  and  the  HF  community. 

4.2.2  Features  of  the  Hypermedia  Information  System 

The  Hypermedia  Information  System  (HIS)  was  designed  with  a  variety  of  features  to  aid  a  reader  of  the 
document  in  locating  and  using  the  information  contained  in  the  system.  Most  of  these  features  were 
derived  by  analyzing  the  eventual  information  needs  of  the  readers  of  the  HIS.  as  well  as  reviewing 
state-of-the-art  information  retrieval  and  hypermedia  systems.  The.se  features  are  discussed  in  the 
subsequent  sections. 


Chapter  Four 


4.22.1  Information  Browsing 

The  HIS  allows  readers  to  browse  through  the  information  contained  in  the  on-line  database.  That  is,  the 
user  is  allowed  to  leisurely  wander  through  the  information,  selecting  items  of  interest  and  inspecting 
various  topics  that  pique  the  interest  of  the  reader.  For  example,  some  readers  might  browse  the 
information  looking  only  at  the  pictures  that  may  have  some  relevancy  to  the  topic  at  hand.  Other  readers 
might  choose  to  view  only  the  titles  and  authors  of  individual  papers,  searching  for  a  topic  that  may  have 
some  application  to  their  current  task. 

The  HIS  supports  this  browsing  operation  in  two  ways.  First,  the  information  is  loosely  structured  in  the 
database,  allowing  the  HIS  to  present  the  same  information  in  many  different  ways.  Second,  the  HIS 
provides  mechanisms  to  the  readers  in  which  they  can  easily  locate  additional  information  related  to  the 
current  topic.  For  example,  while  reading  a  conference  paper  stored  in  the  HIS,  a  reader  might  come 
across  a  reference  to  a  related  photograph.  The  HIS  allows  the  reader  to  select  this  reference  to  view  the 
actual  photograph. 

This  method  of  linking,  or  associating,  information  can  easily  extend  to  other  objects  stored  in  the  HIS. 
For  example,  readers  can  point  to  references  to  videos,  animations,  text,  sound,  etc.  and  request  to  see  (or 
hear)  this  information.  The  HIS  is  able  to  retrieve  these  items  and  display  them  to  the  reader.  This 
linking  capability  allows  the  reader  to  freely  move  about  the  information  contained  in  the  HIS,  literally 
wandering  and  exploring  at  will. 

The  HIS  supports  linking  through  the  use  of  buttons  and  icons  (see  Figure  4.2).  Buttons  allow  the  reader 
to  activate  certain  links,  while  icons  provide  some  sort  of  visual  clue  to  the  reader  as  to  what  the  button 
will  do  when  it  is  activated.  Usually  icons  are  a  picture  of  some  object  (e.g.,  a  video  camera  to  represent 
video).  However,  text  can  also  serve  as  an  icon  (e.g.,  the  words  FIGURE  62  represent  the  actual  figure). 
The  HIS  supports  both  text  and  graphics  as  icons  to  support  the  reader  in  determining  when  a  button  is 
present,  and  what  that  button  will  do  when  it  is  activated.  The  HIS  also  contains  an  overview  diagram 
of  the  information  contained  in  the  database  (Figure  4.3).  This  overview  diagram  allows  a  reader  to  view 
the  entire  HIS  database  at  once.  Using  the  overview  diagram,  the  reader  can  gain  a  perspective  of  how 
one  piece  of  information  is  related  to  other  information.  Also  from  the  overview,  the  reader  can  select 
any  piece  of  information  and  go  directly  to  it,  without  having  to  browse  through  the  system. 

4.2.2.2  Information  Searching 

Often  a  reader  will  come  to  the  HIS  with  a  specific  idea  of  what  information  is  needed.  The  chances  of 
locating  the  exact  piece  of  needed  information  simply  by  browsing  are  fairly  low,  even  in  relatively  small 
documents.  Even  if  a  reader  happens  to  come  across  one  piece  of  relevant  information,  there  might  be 
additional  information  that  the  reader  might  miss.  An  even  worse  scenario  is  when  a  reader  has  a  request 
for  information  that  is  not  contained  in  the  HIS.  Tlie  reader  might  browse  indefinitely  without  finding 
a  relevant  piece  of  information.  The  reader  would  not  be  able  to  tell  if  their  lack  of  success  was  due  to 
their  bad  luck  in  browsing  or  to  the  fact  that  the  information  did  not  exist  at  all. 

To  overcome  these  problems,  the  HIS  provides  a  facility  for  conducting  direct  searches  for  information. 
The  HIS  contains  an  index  into  the  information  that  aids  readers  by  allowing  them  to  search  the  index, 
rather  than  the  information  for  a  piece  of  information.  The  index  contains  both  the  search  items  and  the 
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location  of  those  items  in  the  HIS 
system.  This  technique  is  similar  to 
the  uses  one  might  find  in  the  local 
telephone  listings.  For  example  to 
locate  the  residence  of  your  friend 
Timothy  Handson,  you  could  use  the 
telephone  listings  to  locate  all  of  the 
entries  for  Timothy  Handson.  Once 
you  have  found  the  correct  entry,  you 
are  able  to  get  the  address  of  his 
residence  and  then  go  to  his  home. 

This  directed  search  method  is  much 
more  efficient  than  browsing. 

The  HIS  system  works  in  a  fashion  to 
the  telephone  listing  analogy.  The  HIS 
system  contains  many  indices 
indicating  exactly  what  each  piece  of 
information  is,  including  what  type  of 
information  it  is  (i.e.,  text,  video, 

graphics,  sound),  and  where  it  is  Figure  4.2  Buttons  and  icons  allow  readers  to  access  related 
located  (i.e.,  exactly  what  conference  information 

paper,  what  paragraph,  and  what 
sentence).  Readers  using  the  HIS 
would  enter  a  description  of  the 
information  that  they  are  looking  for, 
and  the  HIS  uses  these  indices  to  tell 
the  reader  where  this  information  is 
located.  The  reader  can  directly  "jump" 
to  this  information,  bypassing  the 
entire  browsing  process. 

In  the  present  HIS  system,  the  reader 
initiates  a  search  by  entering  a  query, 
or  a  description  of  the  desired 
information,  into  the  system  (Figure 
4.4).  This  query  is  made  up  of 
individual  terms  and  boolean  operators 
on  those  terms.  For  example,  search 
terms  could  include:  Airplane.  Human 
Factors.  Helicopter.  Navigation,  etc.  4.3  The  Overview  Diagram 

Any  word  is  a  valid  search  term.  The 

reader  also  uses  boolean  operators  (AND,  OR,  NOT)  to  indicate  relationships  between  multiple  .search 
terms.  For  example,  a  possible  query  could  be  Computer  AND  Training,  indicating  that  reader  is  looking 
for  a  piece  of  information  that  is  related  to  both  computers  and  training. 


HF  Issues  Aviation  Maintenance 
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The  HIS  uses  this  query  to  search  the 

indices  for  related  information.  In  the  ^||||||||||||||||[|||||||||||||^^ 

case  of  the  example  query  above,  the 

HIS  would  first  look  for  information  training  and  computers  and  airplanes 
related  to  computers,  then  look  for 
information  related  to  training,  and 

would  then  look  for  pieces  of  - ^ 

information  that  were  in  both  of  those  V/hal  text  are  you  searching  for? 

wsets.  The  results  of  this  search  are 
then  presented  back  to  the  reader  (see 

Figure  4,5).  Often,  there  are  multiple  Figure  4.4  Entering  Search  Terms 
pieces  of  information  that  match  a 

readers  query  (e.g.,  just  as  there  might  be  multiple  listing  for  Timothy  Handson  in  the  telephone  book). 
In  this  case,  the  readers  are  shown  all  of  these  matching  pieces  of  information,  and  are  allowed  to  browse 
through  those  that  they  feel  could  be  relevant. 

4.2.2 J  Search  Aiding 


Text  Search 


training  and  computers  and  airplanes 


o 


What  text  are  you  searching  tor? 


Figure  4.4  Entering  Search  Terms 


When  the  HIS  finds  multiple  matches 
to  a  query,  these  matches  are  presented 
to  the  reader  for  review.  The  HIS  also 
aids  the  reader  in  determining  which 
matches  are  likely  to  be  most  relevant 
to  their  query  by  providing  relevancy 
bars  along  with  the  query  results  (see 
Figure  4.5).  The  magnitude  of  each 
relevancy  bar  indicates  the  likelihood 
of  that  document  being  most  relevant; 
the  longer  that  bar,  the  more  relevant 
the  document  is  likely  to  be  to  the 
reader. 

Another  search  aiding  technique  is 
allowing  the  reader  to  refine  and  edit 
previous  queries.  Entering  a  search 
query  is  usually  not  a  one-time 
process.  Sometimes  a  query  is  too 
broad,  in  that  too  many  pieces  of 
information  are  retrieved  for  the  reader  to  handle  at  a  time.  Sometimes,  a  query  is  too  narrow,  yielding 
little  or  no  matches  to  the  query.  Readers  can  refine  queries  either  by  adding  or  deleting  search  tc’-ms. 
The  HIS  retains  previous  queries  to  aid  the  reader  in  determining  which  queries  to  expand. 


Figure  4.5  Search  results  are  shown  on  the  overview 


diagram  by  highlighting  matching  items 


58 


The  FAA  Aviation  Maintenance  Human  Factors  Hypermedia  System 


The  HIS  also  aids  the  reader  by  expanding  and  refining  queries.  To  be  implemented  in  future  versions 
of  the  HIS,  a  term  thesaurus  will  be  included  to  help  the  reader  in  locating  additional  terms  that  might  be 
relevant  to  the  desired  information.  These  terms  can  the  be  used  by  the  reader  as  part  of  subsequent 
queries  into  the  information. 

4.23  System  Architecture 

The  HIS  operates  on  an  IBM  PC-compatible  desktop  platform  with  4  Megabytes  of  system  memory,  a 
Microsoft  Mouse  (or  compatible),  and  Microsoft  Windows  3.1.  Initial  versions  of  the  HIS  can  run  off  of 
the  system’s  hard  disk,  but  future  versions  will  require  a  CD-ROM  reader  to  use  the  system,  due  to  the 
targe  amount  of  information  contained  in  the  system.  The  overall  software  architecture  is  shown  in  Figure 
4.6.  The  system  was  constructed  using  a  variety  of  development  tools,  including:  Microsoft  Windows  3.1, 
Asymetrix  Toolbook,  Borland  C-n-  Compiler,  and  Raima  Corporation  DB_Vista. 

These  tools  were  used  to  construct  two 
distinct  environments.  The  first 
environment  is  the  reader  interface, 
which  was  described  in  earlier  sections. 

The  reader  interface  allows  users  of  the 
information  to  search  and  browse  the 
information  contained  in  the  HIS.  The 
second  environment,  the  author 
interface,  contains  tools  that  allow 
authors  to  place  information  into  the 
HIS.  These  tools  include  editors,  link 
builders,  indexers,  and  database 
manipulation  tools.  Future  versions  of 
the  HIS  will  allow  access  to  these  tools 
from  the  reader  interface,  thus  allowing 
readers  to  store  and  index  information  in  addition  to  the  material  already  contained  in  the  HIS. 

The  HIS  contains  two  distinct  databases.  The  first  database,  the  Media  Database,  contains  the  actual 
information  to  be  retrieved  by  the  user.  This  material  includes  graphics,  text,  and  eventually  sound, 
animations,  and  other  multimedia  information.  The  second  database,  the  Link/Index  Database,  contains 
information  to  help  the  HIS  search  for  and  retrieve  information.  The  Link/Index  database  contains 
information  regarding  file  location,  search  term  location,  document  information,  graphic  information,  and 
link  structures. 

4.3  CURRENT  RESEARCH  STATUS  AND  FUTURE  PLANS 

Phase  I  has  produced  a  working  prototype  of  HIS.  It  includes  an  on-line  version  of  the  3rd  FAA  Human 
Factors  in  Aviation  Maintenance  meeting.  This  prototype  is  available  for  distribution.  Directions  for  future 
research  include: 


Figure  4.6  The  Overall  Architecture  of  HIS 
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1)  Provide  one  operation  for  indexing  the  HIS  information.  Presently,  three  distinct 
operations  are  required  to  store  this  information.  By  integrating  these  three  steps  into  a 
common  environment,  these  three  operations  can  be  performed  simultaneously. 

2)  Improve  navigational  strategies  to  allow  readers  to  return  to  information  already  seen. 
Similar  to  the  idea  of  "bookmarks,"  this  strategy  will  allow  readers  to  quickly  mark  and 
return  to  important  information. 

3)  Improve  searching  strategies  by  providing  feedback  to  the  users  regarding  the 
effectiveness  of  their  searches.  Aids  such  as  on-line  thesauruses,  enhanced  search  control, 
and  relevancy  feedback  can  be  used  to  improve  searching  by  even  the  most  novice  user. 

4)  Integrate  the  authoring  and  reader  interfaces.  This  integration  will  allow  future  users  of 
the  HIS  to  incorporate  relevant  information  into  the  HIS  system  and  will  allow  them  to 
rapidly  associate  this  information  with  material  already  stored  in  the  HIS. 

5)  Incorporate  additional  media  types  such  as  video,  animations,  and  sound. 

The  technology  provided  by  this  research  will  be  used  to  support  the  information  retrieval  needs  across 
a  variety  of  FAA  and  airline  maintenance  software  support  systems.  Already,  various  training  systems 
have  benefitted  from  incorporating  the  HIS  technology  into  these  systems.  Students  are  now  able  to 
quickly  access  information  when  it  is  needed  during  a  training  session.  Current  plans  include  the 
incorporation  of  the  HIS  technology  into  the  various  maintenance  job  aiding  systems  as  well.  Anticipated 
benefits  of  this  marriage  between  hypermedia  and  Job  aiding  include  quicker  access  to  information, 
exposure  to  material  that  might  have  been  overlooked  using  conventional  information  access  systems,  and 
subsequent  reduction  of  maintenance  personnel  error. 
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Chapter  Five 

Human  Reliability  in  Aircraft  Inspection 

5.0  INTRODUCTION 

This  section  describes  the  continuing  work  on  aircraft  inspection,  whose  long-term  objective  is  to  enhance 
system  reliability  through  human  factors  interventions.  It  builds  upon  the  Phase  I  outcomes  reported  in 
Shepherd,  et  al.,  1991,  and  thus  does  not  re-Justify  human  factors  applications  in  this  field. 

Phase  I  provided  detailed  Task  Descriptions  and  Task  Analyses  of  many  aircraft  inspection  activities 
observed  at  major  carriers  in  the  U.S.A.  During  Phase  II,  visits  were  made  to  other  inspection  sites,  with 
coverage  of  regional  airlines,  repair  centers,  and  sites  in  the  U.K.  (see  Section  5.3.6).  TThe  concentration 
was  on  specific  aspects  of  the  system,  such  as  Non-Destructive  Inspection  (NDI),  information  flow,  and 
training.  Although  inspection  tasks  were  observed,  no  additional  formal  Task  Analyses  are  reported  here. 

In  Phase  II,  the  implications  of  the  data  collected  earlier  have  been  researched  in  more  detail  than  was 
provided  in  Shepherd,  et  al.,  1991.  This  has  led  to  a  series  of  studies  by  the  research  team,  all  under  the 
objective  of  human  factors  interventions  to  improve  inspection  system  reliability.  These  studies  can  be 
broadly  classified  into  those  with  short-term  and  long-term  outcomes.  While  the  former  have  led  to 
specific,  on-going  interventions  at  airline  inspection  sites,  the  tatter  have  produced  insights  and  on-going 
experiments  in  an  off-site  setting.  One  additional  activity  has  been  a  joint  project  with  the  Civil 
Aeronautics  Authority  (CAA)  in  the  U.K.  to  document  and  evaluate  international  differences  in  civil 
aircraft  inspection  (Drury  and  Lock,  1992). 

Chapter  3  of  Shepherd,  et  al.,  1991  listed  a  set  of  short-term  and  long-term  research  needs,  and  this  list 
has  provided  the  guidance  for  Phase  II  work.  AH  of  these  needs  were  derived  from  a  basic  description 
of  the  inspection  system,  and  a  generic  task  description  of  inspection.  As  these  descriptions  form  the  basis 
of  all  that  follows,  an  updated  system  description  (from  Drury  and  Lock.  1992)  is  included  here. 

5.1  THE  INSPECTION  SYSTEM:  A  HUMAN-FACTORS  DESCRIPTION 

An  aircraft  structure  is  designed  to  be  used  indefinitely  provided  that  any  defects  arising  over  time  are 
repaired  correctly.  Most  structural  components  do  not  have  a  design  life,  but  rely  on  periodic  inspection 
and  repair  for  their  integrity.  There  are  standard  systems  for  ensuring  structural  safety  (e.g.,  Goran.son  and 
Miller,  1989),  but  the  one  which  most  concerns  us  is  that  which  uses  engineering  knowledge  of  defect 
types  and  their  time  histories  to  specify  appropriate  inspection  intervals.  The  primary  defects  are  cracks 
and  corrosion  (which  can  interact  destructively  at  times)  ari.sing  respectively  from  repeated  stretching  of 
the  structure  from  aerodynamic  or  internal  pre.ssure  loads,  and  from  weathering  or  harmful  chemicals. 
Known  growth  rates  of  both  defect  types  allow  the  analyst  to  choose  intervals  for  in.spection  al  which  the 
defects  will  be  both  visible  and  safe.  Typically,  more  than  one  such  inspection  is  called  for  between  the 
visibility  level  and  the  safety  level  to  ensure  some  redundancy  in  the  inspection  process.  As  the  inspection 
system  is  a  human/machine  system,  continuing  airworthiness  has  been  redefined  by  the  design  process 
from  a  mechanical  engineering  problem  to  an  ergonomic  one.  Inspection,  like  maintenance  in  general, 
is  regulated  by  the  FAA  in  the  U.S.A.,  the  CAA  in  the  U.K.,  and  equivalent  bodies  in  other  countries. 
However,  enforcement  can  only  be  of  following  procedures  (e.g.,  hours  of  training  and  record-keeping  to 
show  that  tasks  have  been  completed),  not  of  the  effect ivene.ss  of  each  inspector.  Inspection  is  al.so  a 
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complex  socio-technical  system  (Taylor,  1990),  and  as  such,  can  be  expected  to  exert  stresses  on  the 
inspectors  and  on  other  organizational  players  (Drury,  1985). 

Maintenance  and  inspection  are  scheduled  on  a  regular  basis  for  each  aircraft,  with  the  schedule  eventually 
being  translated  into  a  set  of  job  cards  for  the  aircraft  when  it  arrives  at  the  maintenance  site.  Equipment 
which  impedes  access  is  removed  (e.g.,  seats,  galleys).  The  aircraft  is  cleaned,  and  access  hatches  are 
opened.  Next  comes  a  relatively  heavy  inspection  load  to  determine  any  problems  (cracks,  corrosion, 
loose  parts)  which  will  need  repair.  During  inspection,  each  of  these  inspection  findings  is  written  up  as 
a  Non-Routine  Repair  (NRR)  item.  After  some  NRRs  are  repaired,  an  inspector  must  approve  or  "buy 
back"  these  repairs.  Thus,  the  workload  of  inspectors  is  very  high  when  an  aircraft  arrives  (often 
necessitating  overtime  working),  decreases  when  initial  inspection  is  complete,  and  slowly  increases 
towards  the  end  of  the  service  (due  to  buybacks).  Much  of  the  inspection  is  carried  out  in  the  night  shift, 
including  routine  inspections  on  the  flightline,  of  aircraft  between  the  last  flight  of  the  day  and  first  flight 
of  the  next. 

At  a  more  detailed  level,  the  task  of  inspection  can  be  broken  into  a  set  of  subtasks  which  follow  in 
logical  order.  Table  5.1  shows  a  generic  task  description  based  on  simpler  tasks  for  industrial  inspection 
tasks  (Drury,  1978).  For  each  subtask.  Table  5.1  presents  an  example  from  both  Visual  Inspection  and 
Non-Destructive  Inspection  (NDI).  In  a  typical  inspection  schedule,  well  over  90%  of  the  job  cards  are 
for  Visual  Inspection. 

With  these  seven  task  steps,  the  complex  problems  of  error  control,  design  of  the  information  environment, 
and  development  of  training  schemes  all  become  more  manageable  as  specific  human  factors  knowledge 
can  be  brought  to  bear  on  each  task  step  in  turn.  The  current  review  of  projects  shows  this  structure 
clearly,  both  in  terms  of  deriving  the  needs  for  rapid  interventions,  and  in  developing  off-line  experiments 
to  investigate  the  sensitivity  of  human  performance  to  systems  variables. 

5.2  SHORT-TERM  DEMONSTRATION  PROJECTS 

Although  human  factors  engineering  is  becoming  known  to  the  aviation  maintenance  community  through 
the  FAA/AAM  series  of  meetings,  there  is  still  a  need  to  show  straightforward,  practical  interventions 
which  produce  relatively  rapid  changes.  Such  demonstration  projects  can  lead  to  widely  disseminated 
changes,  and  to  a  model  for  how  human  factors  studies  can  be  conducted  by  airlines  themselves.  Three 
projects  were  chosen  by  FAA/AAM,  of  which  two  were  to  be  pursued  during  Phase  II,  with  the  choice 
left  to  the  airlines  themselves.  The  three  projects  were  on  redesign  of  hard-copy  workcards  (job  cards), 
design  of  the  lighting  environment  for  inspection,  and  redesign  of  the  human  interface  of  typical  NDI 
equipment  to  follow  human  factors  principles.  The  first  two  of  these  to  be  taken  up  by  the  industry  were 
the  workcards  and  lighting  projects,  so  these  are  described  in  some  detail  in  the  following  sections.  Other 
projects,  including  the  NDI  interface  design,  are  to  be  performed  in  future  years  and  hence  are  described 
briefly. 

To  our  knowledge  one  aircraft  manufacturer  and  one  airline  company  have  started  human  factors  groups 
in  the  maintenance/inspeciion  field,  but  this  still  leaves  many  other  airlines  with  a  shortage  of  human 
factors  expertise.  Information  is  available  through  the  proceedings  of  the  FAA/AAM  meetings  on  Human 
Factors  in  Aircraft  Maintenance  and  Inspection,  but  it  is  often  either  human  factors  specialists  telling  what 
could  be  done,  or  existing  industry  personnel  showing  what  has  been  done  without  formal  human  factors 


64 


Human  Reliability  in  Aircraft  Inspection 


knowledge.  With  this  background  the  short-term  demonstration  projects  have  been  structured  to  allow 
human  factors  specialists  and  aircraft  industry  personnel  to  work  together  on  projects  which  neither  could 
conveniently  perform  alone.  To  this  end,  the  FAA/AAM  support  has  provided  human  factors  expertise, 
while  airline  partners  have  provided  facilities  and  personnel  with  detailed  knowledge  of  inspection  of 
particular  aircraft.  The  airline  partners  have  also  agreed  to  provide  travel  to  and  from  the  work  site.  For 
their  cooperation,  airline  partners  get  their  personnel  to  understand  some  aspects  of  human  factors,  as  well 
as  a  response  to  their  specific  needs.  Alt  partners  have  agreed  to  allow  dissemination  of  study 
methodology  and  results. 


TASK  DESCRIPTION 

VISUAL  EXAMPLE 

NDT  EXAMPLE 

1.  Initiate 

Get  workcard.  Read  and  understand  area 
to  be  covered. 

Get  workcard  and  eddy 
current  equipment.  Calibrate. 

2.  Access 

Locate  area  on  aircraft.  Get  into  correct 
position. 

Locate  area  on  aircraft. 

Position  self  and  equipment. 

3.  Search 

Move  eyes  across  area  systematically. 

Stop  if  any  indication. 

Move  probe  over  each  rivet  head.  Stop  if  any 
indication. 

4.  Decision 

Making 

Exantine  indication  against  remembered 
standards,  e.g.,  for  dishing  or  corrosion. 

Re-probe  while  closely  watching  eddy  current 
trace. 

5.  Respond 

Mark  defect.  Write  up  repair  sheet  or  if  no 
defect,  return  to  search. 

Mark  defect.  Write  up  repair  sheet  or  if  no 
defect,  return  to  search. 

6.  Repair 

Drill  out  and  replace  rivet. 

Drill  out  rivet.  NDT  on  rivet  hold.  Drive  out  for 
oversize  rivet. 

7.  Buy-back  Inspect 

Visually  inspect  marked  area. 

Visually  inspect  marked  area. 

Table  5.1  Generic  Task  Description  of  Incoming  Inspection,  with  Examples  from  Visual  and  NDT 
Inspection 


As  these  are  on-going  projects,  with  the  first  two  due  for  completion  in  May  1992,  only  the  needs  and 
methodology  are  presented  here. 

5.2.1  HUMAN  FACTORS  IN  WORKCARD  DESIGN 

A  major  air  carrier  has  agreed  to  become  the  partner  on  the  workcard  design  project,  working  through 
maintenance  facilities.  Although  the  issue  of  information  flow  within  the  in.spection/maintenance  system 
is  complex  (see  Section  5.3 J),  and  high-technology  interventions  are  possible  (Johnson,  1990),  many 
airlines  have  too  large  an  investment  in  current  hardware  to  consider  alternatives  beyond  hard-copy 
workcards  as  the  inspectors’  primary  information.  Airlines  often  have  computer-generated  workcards,  and 
wish  to  continue  using  some  version  of  the  same  medium,  at  least  in  the  near-term.  Thus,  while  we  are 
moving  towards  new  generations  of  computer-based  job  information  aids,  there  is  still  an  on-going  need 
to  apply  human  factors  techniques  to  existing  workcard  generation  systems. 


The  workcard  controls  the  inspection  workllow  by  describing  to  the  inspector  the  location  of  the  work 
area,  the  area(s)  to  be  inspected,  and  the  inspection  procedure.  It  is  the  primary  document  that  inspectors 
carry  during  inspection. 
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The  task  analyses  ol'aircral't  inspxjction  (Drury,  Prabhu  and  Gramopadhye,  1990)  suggested  that  workcards 
are  the  main  source  of  on-line  feedforward  information.  However,  even  within  the  relatively  homogeneous 
sample  of  air  carriers,  there  was  considerable  variability  in  the  design  of  these  documents.  Since  the 
"paper  document"  is  currently  the  prevalent  and  preferred  means  by  which  the  inspector  has  access  to  the 
information  that  is  needed  on  the  job,  the  availability  of  quality  documentation  is  of  critical  importance 
to  inspection  performance. 

Table  5.2  classifies  the  various  human  factors  issues  which  the  Task  Analysis  data  showed  to  be  relevant 
to  documentation  design.  The  workcard,  which  is  a  paper  document,  must  be  evaluated  with  these  issues 
in  mind.  The  taxonomy  also  provides  a  framework  with  which  to  design  a  new  workcard  which  adheres 
to  human  factors  principles. 


1 .  Information  layering 

•  Amount  of  information  •  Levels  of  information 

•  Experts  versus  novices  •  Accessibility 

•  Use  of  cues/indications/checklists  •  Flexibility  of  use 

2.  Layout  of  information 

•  Spatial  layout 

•  Grouping  of  information 

•  Chunking 

3.  Presentation  of  Text 

•  Visual  organization  •  Word  spacing 

•  Visual  density  •  Line  spacing 

•  Letter  case  •  Line  length 

4.  Presentation  of  Graphics 

•  Visual  organization  •  Contrast 

•  Spatial  location  w.r.t.  text  •  Labeling 

5.  Language  Constraints 

•  Minimal  number  of  words  •  Fixed  syntax 

•  Standardized  nomenclature  •  Concise  wording 

•  Appropriate  abbreviations 

6.  Physical  Implementation 

•  Completeness  of  information  •  Physical  accessibility 

•  Consistency  across  cards  •  Accuracy  of  information 

Table  5.2  A  Taxonomy  of  Human  Factors  Issues  in  Workcard  Design 

Since  the  workcard  is  the  means  of  communication  of  command  information  (both  directive  and 
feedforward),  it  is  important  to  understand  the  effects  of  workcard  design  on  the  use  of  its  information 
content  by  the  inspectors.  Current  research  in  human  factors  and  cognitive  science  in  the  areas  of 
information  processing,  visual  perception,  learning,  document  design  and  computer  display  design  (e.g., 
Wright,  1991)  provide  us  theoretical,  as  well  as  empirical,  guidelines  that  can  be  used  for  the  design  of 
more  effective  workcards.  The  taxtmomy  is  an  attempt  to  organize  these  guidelines  to  provide  a 
framework  that  can  direct  the  documentation  design  process. 

Table  S3  presents  an  analysis  of  the  original  Task  Analysis  data  of  aircraft  inspection,  classified  using 
the  above  taxonomy.  The  points  rai.sed  are  not  in  any  implied  order  of  importance. 
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Information  Layering 

•  None  of  the  workcards  provided  layered  information,  i.e.,  the  opportunity  to  access  more 
detailed  information  on  inspector  selected  points. 

•  Key  points  were  not  clearly  differentiated. 

•  Little  feedforward  on  problem  areas  to  be  expected  on  this  aircraft  at  this  time. 

•  Few  workcards  provided  checklist  of  probable/possible  defects. 

•  Necessary  safety  precautions  not  specified. 

•  Task  cards  did  not  specify  limits  on  wear,  play,  etc.  for  visual  inspection  to  help  the  human 
inspector  make  more  consistent  judgements. 

•  Cues  indicating  defects  not  listed  in  workcards,  e  g.  scuffed  paint  on  fairings  (of  wing)  indicates 
rubbing. 

Layout  of  Information 

•  Poor  legibility  in  some  workcards. 

•  Relevant  information  spread  over  multiple  pages. 

•  Much  information  of  a  legal  or  general  nature  occupied  pnme  space  at  the  top  of  the  card. 

•  Column  layout,  text  boxes  and  other  enhancements  now  available  on  computer-generated  text 
were  not  used. 

Presentation  of  Text 

•  Soma  cards  were  in  all  capitals,  a  violation  of  human  factors  principles. 

•  Font  design  was  not  considered  for  legibility  in  highly  variable  lighting  conditions. 

•  Quality  of  printing  and  copying  was  not  uniformly  good. 

Presentation  of  Graphics 

•  Workcards  designed  without  considering  correct  location  of  graphics. 

•  Some  graphics  were  confusing  even  to  experienced  inspectors. 

•  Some  graphics  for  accessing  the  inspection  area  were  ambiguous. 

•  Most  graphics  were  of  poor  quality. 

•  Color  coding  was  not  used,  but  may  need  to  be  considered  if  it  offers  worthwhile  performance 
improvements. 

Language  Constraints 

•  Procedures  were  not  concisely  worded  in  many  workcards. 

•  There  seems  to  be  no  evidence  of  any  conscious  design  procedure  to  use  fixed  syntauc, 
consistent  use  of  phrases  or  a  standardized  nomenclature. 

Physical  Implementation 

•  Imperfect  matching  of  nomenclature  for  parts  and  defects  between  worksheet  and  secondary 
source  material. 

•  Inconsistent  description  of  tasks.  Some  were  described  very  briefly  and  others  in  detail. 

•  No  tool  or  aid  that  ensures  that  the  inspector  has  covered  the  entire  inspection  area. 

•  Drawings  on  workcard  sometimes  do  not  match  configuration  of  the  same  area  on  the 
workcard. 

•  Illumination  specifications  not  available/not  specified  on  workcard. 

•  Some  workcards  do  not  have  figures  showing  relative  location  of  various  parts, 

•  Some  workcards  did  not  specify  equipment/gauges  to  be  used  in  the  inspection. 

•  Completeness  and  currency  of  information  is  not  assured,  and  not,  therefore,  trusted  by  all 
inspectors. 

•  Physical  size  and  shape  of  workcards  is  not  always  well  integrated  with  other  tools  the 
inspector  must  carry  and  use. 

Table  5.3  Classification  of  Observations  from  the  Task  Analyses  of  Inspection,  Classified  by  the 
Taxonomy  of  Table  5.2 


5.2.1. 1  A  Demonstration  Proj>ram  for  Workcard  Redesign 

With  our  airline  partner,  a  workcard  redesign  program  is  being  undertaken  as  a  demonstration  of  how 
human  factors  techniques  can  improve  inspection.  Hxi.sting  workcards  for  a  small  number  of  relatively 
common  maintenance  events  (an  A-check  and  a  C-check)  are  being  analyzed  with  respect  to  the  i.ssues 
derived  in  the  taxommiy.  Oood  and  poor  aspects  of  the  workcard  design  have  been  noted,  both  from 
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analysis  of  the  workcard  itself  and  from  analysis  of  its  use  by  inspectors.  From  this  data  collection  phase 
will  come  a  series  of  design  requirements  which,  if  met,  will  ensure  good  human  factors  design. 

With  airline  partner  representatives,  design  solutions  will  be  developed  to  cover  both  short-term  and 
long-term  changes.  Short-term  interventions  for  workcards  may  include,  for  example; 

1.  Changing  the  presentation  format  and  layout  to  improve  ease  of  use  and  legibility. 

2.  Ensuring  that  visual  material  is  incorporated  into  the  worksheet. 

3.  Consistent  naming  of  parts,  directions,  defects,  and  indications  between  all 
documents  used  by  inspectors. 

4.  Multi-level  workcard  systems,  useable  by  inspectors  with  different  levels  of 
immediate  familiarity  with  the  worksheet  content. 

5.  A  better  physical  integration  between  the  workcard  and  the  inspector’s  other 
documents  and  tools  needed  at  the  worksite. 

6.  Providing  a  better  spatial  integration  between  the  workcard  and  the  inspection  tasks 
around  the  aircraft. 

Each  design  solution  will  be  implemented  and  a  series  of  prototype  workcards  produced.  These  will  be 
pre-tested  by  having  inspectors  use  them  while  providing  a  verbal  protocol  of  their  actions.  From  this  user 
evaluation  will  come  a  refined  design. 

The  final  design  will  be  tested  against  the  current  design  using  controlled  tests  during  A-checks  and 
C-checks.  Measurements  will  be  taken  of  inspector  verbal  protocols,  errors/confusions  observed,  and 
questionnaire  evaluation  from  both  inspectors  and  supervisors. 

The  results  will  be  documented  as  a  case-study  to  show: 

a.  How  other  maintenance/inspection  operations  can  improve  their  workcards. 

b.  How  to  apply  human  factors  principles  to  the  improvement  of  other 
maintenance/inspection  functions. 

5.2.2  DESIGNING  THE  VISUAL  ENVIRONMENT  FOR  INSPECTION 

A  second  major  carrier  is  cooperating  with  the  University  at  Buffalo  team  to  improve  the  iaspector’s  visual 
environment.  This  project  is  based  at  the  maintenance  facilities  operated  by  the  carrier  at  a  single  airport. 
There  is  a  single  maintenance  hangar,  with  three  aircraft  bays,  and  apron  areas  outside  the  hangar  and  by 
the  gates.  The  main  concentration  will  be  on  in-hangar  activities,  but  other  sites  will  also  be  considered. 
Having  a  single  hangar  makes  the  demonstration  project  manageable  while  still  providing  a  representative 
application  of  human  factors. 

Analysis  of  aircraft  inspection  activities  has  shown  that  visual  inspection  dominates  other  inspection 
activities  (Drury,  Prabhu,  and  Gramopadhye,  1990).  Since  visual  inspection  is  such  an  important 
component,  accounting  for  almost  90%  of  all  inspection  activities,  it  is  imperative  that  the  task  be 
performed  in  the  most  suitable  work  environment.  From  the  task  analy.sis  of  various  inspection  tasks  in 
Table  5.1,  it  is  seen  that  "visual  search"  is  an  important  component  of  the  in.spection  task,  and  the  success 
of  this  stage  is  critical  for  successful  completion  of  tlie  inspection  task.  In  visual  search  the  in.spector  mu.st 
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closely  examine  each  area  for  a  list  of  potential  faults.  The  amount  of  effort  required  on  the  part  of  the 
inspector  for  each  area  depends  upon  various  factors  such  as  the  prior  information  (from  training 
experience  on  the  workcard)  and  the  suitability  of  the  physical  conditions  for  inspections  (lighting, 
illumination  levels,  etc.). 

Studies  in  aircraft  inspection  have  shown  that  poor  illumination,  glare,  and  other  adverse  lighting 
conditions  could  be  the  single  most  important  reason  for  "eye  strain"  or  visual  fatigue.  Visual  fatigue 
results  in  deterioration  in  the  efficiency  of  human  performance  during  prolonged  work.  Progressively 
more  effort  is  required  to  maintain  performance,  and  eventually  performance  level  decreases  despite  the 
extra  effort.  The  purpose  of  this  study  is  to  identify  potential  sources  of  improvement  in  ins  action 
lighting  and  to  suggest  modifications  so  that  the  task  can  be  performed  under  improved  visual  conditions. 

From  the  detailed  Task  Analyses  of  numerous  inspection  activities  performed  in  Phase  I,  Table  5.4  gives 
a  list  of  examples  of  poor  human  factors  design.  Each  represents  an  opportunity  for  intervention  to 
improve  the  human/system  fit  and  hence,  increase  job  performance  with  decreased  work  stress. 


1. 

Illumination  levels  beneath  the  aircraft  varied  depending  on  the  location.  Measured  levels  during  daylight  ranged  from 

25  ft.  candles  under  wings  and  fuselage  areas  on  the  side  of  the  aircraft  facing  open  hangar  doors,  to  2-5  ft.  candles 
under  the  wings  and  fuselage  on  the  opposite  side  of  the  aircraft. 

2. 

Lighting  levels  beneath  the  aircraft  were  adequate  only  for  gross  visual  inspection. 

3. 

Inspectors  often  used  a  flashlight  as  an  aid  during  visual  inspection.  It  should  be  noted,  however,  that  the  type  of 
flashlight  used  was  not  consistent  among  all  inspectors  even  within  a  single  aircraft  operator,  and  varied  considerably 
between  carriers. 

4. 

Often  it  was  found  that  the  overhead  lighting  used  for  general  illumiration  was  covered  with  dirt  and  paint,  inhibiting  full 
illumination  capability. 

5. 

Aircraft  near  the  hangar  doors  were  exposed  to  higher  illumination.  Illumination  ranged  from  16  ft.  candles  to  1 14  ft. 
candles. 

6. 

Lighting  conditions  were  different  during  the  day  shift  and  night  shift.  During  the  night  shift  illumination  ranged  from 

29-33  ft.  candles. 

7. 

Supplemental  lighting  was  often  not  provided  under  the  aircraft.  Measured  levels  in  these  areas  ranged  from  a  day  time 
high  of  42  ft.  candles  to  a  night  time  low  of  1 .2  ft.  candles. 

8. 

The  general  lighting  level  inside  the  aircraft  fuselage  averaged  between  1.5  ft.  candles  to  3  ft.  candles. 

9. 

Glare  from  open  hangar  doors  within  the  inspector’s  visual  field  was  apparent. 

10. 

When  using  supplemental  lighting  (flashlight,  helmet  light,  portable  fluorescent)  the  illumination  on  the  aircraft  surface 
was  irrcreased,  but  glare  was  caused  when  searching  inside  the  aircraft  structure. 

11. 

Portability  of  supplemental  lighting  was  often  poorly  designed,  and  not  well  integrated  into  the  inspector’s  kit. 

Table  5.4  Observations  on  Visual  Environment  of  Inspection  from  Task  Analysis  Data 
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In  designing  lighting  systems,  the  following  factors  need  to  be  considered: 


•  Recommended  Light  Levels  for  Different  Tasks 

The  recommended  illumination  depends  upon  the  type  of  task  and  whether  the  visual  task  is  of  high  or 
low  contrast.  The  Illuminating  Engineering  Society  (lES,  1984)  recommends  that  surface  areas  requiring 
visual  inspection  be  provided  with  75-100  ft.  candles  (800-1050  lux)  of  illumination.  Vision  can  be 
improved  by  increasing  the  lighting  level,  but  only  up  to  a  point,  because  the  law  of  diminishing  return 
operates  (e.g.,  lES  Lighting  Handbook,  New  York,  1984).  Increased  illumination  could  also  result  in 
increased  glare.  Older  persons  are  more  affected  by  the  glare  of  reflected  light  than  younger  people,  and 
inspectors  are  often  senior  personnel  within  an  organization. 

•  Selection  of  Light  Sources  for  Color  Rendering 

In  the  selection  of  artificial  light  sources  one  of  the  mo.st  important  considerations  is  color  rendering,  i.e., 
the  degree  to  which  the  perceived  colors  of  an  object  illuminated  by  various  light  sources  match  the 
perceived  colors  of  the  same  object  when  illuminated  by  a  standard  light  source.  Color  rendering  could 
be  important,  because  often  "change  in  color"  of  true  sheet  metal  is  used  as  a  clue  to  indicate  corrosion. 

•  Direct  and  Indirect  Lighting:  Glare 

The  quality  of  illumination  can  be  improved  by  reducing  glare.  Direct  glare  is  caused  when  a  source  of 
light  in  the  visual  field  is  much  brighter  than  the  task  material  at  the  workplace.  Thus,  open  hangar  doors, 
roof  lights,  or  even  reflections  off  a  white  object  such  as  the  workcard  can  cause  glare  from  surrounding 
surfaces.  Glare  can  be  reduced  by  resorting  to  indirect  lighting.  Of  particular  concern  is  that  in  inspecting 
partially-hidden  areas  (e.g.,  inside  access  panels)  the  lighting  used  to  illuminate  the  defect  may  cause 
glare  from  surrounding  surfaces.  Carefully  designed  combinations  of  general  area  lighting,  portable  area 
task  lighting  and  localized  spotlighting  need  to  be  produced. 

•  Stx;cialized  Lighting 

During  visual  inspection  of  an  aircraft  structure  the  inspector  is  looking  for  multiple  defects,  such  as 
corrosion,  ripples,  hairline  cracks,  dents,  missing  rivets,  damaged  rivets  (e.g.,  "pooched",  "dished"  rivets), 
and  rivet  cracks. 

It  is  possible  that  not  one  single  lighting  system  is  suitable  for  detecting  all  defects.  Therefore,  the  use 
of  a  specialized  lighting  system  for  each  cla.ss  of  defects  may  be  necessary.  However,  the  use  of  special 
light  systems  has  one  major  drawback.  It  implies  that  the  area  mu.st  be  examined  for  each  class  of  defects 
sequentially  rather  than  simultaneously,  which  could  involve  time  and  expense.  A  typical  example  is  the 
difference  between  general  illumination  and  the  grazing  illumination  provided  by  special  purpose  lighting. 
The  diffused  nature  of  general  illumination  tends  to  wa.sh  out  the  shadows  while  the  surface  grazing  light 
relies  upon  showing  shadows  to  emphasize  objects  that  project  above  or  below  the  surface.  Task  visibility 
for  surface  topography  is  distinctly  better  with  grazing  light,  whereas  ct)lor  changes  or  corrosion  may  be 
better  seen  under  general  illumination.  An  example  of  surface  topography  is  the  inspection  of  the  fuselage 
for  ripples.  Ripples  are  easy  to  detect  using  .surface-grazing  light  but  general  illumination  tends  to  wash 
them  out.  However,  .strong  side  lighting  may  ma.sk  important  color  differences. 
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•  Design  Requirements  for  Lighting 

Studies  of  visual  search  have  shown  that  the  speed  and  accuracy  with  which  the  search  process  can  be 
accomplished  is  dependent  on  the  conspicuity  of  the  defect,  which  in  turn  is  dependent  on  the  size  of  the 
defect,  defect/background  contrast,  and  lighting  intensity  (see  Section  5.33). 

Lighting  design  has  a  clear  impact  upon  the  final  two  variables,  but  it  has  broader  requirements  to  fulfill 
as  visual  inspection  involves  more  than  visual  search.  Lighting  should  be  designed  such  that  the  following 
tasks  can  all  be  performed  satisfactorily  and  preferably  optimally: 

1.  Inspection  (visual  search)  of  the  aircraft  fuselage  for  defects. 

2.  Reading  the  workcard/instructions. 

3.  Movement  around  the  aircraft  (using  the  scaffolding,  or  equipment,  e.g.,  cherrypicker). 

In  addition,  special  purpwse  lighting  should  not  interfere  with  any  other  parallel  task  in  progress.  In 
designing  the  visual  environment,  one  must  consider  the  minimum  lighting  requirements  for  each  task  and 
subtask,  the  type  of  artificial  light  sources  that  can  be  used  to  illuminate  the  work  surface,  the  amount  of 
task  lighting  that  can  be  provided,  and  the  available  methods  to  minimize  glare.  These  factors  must  be 
balanced  with  implementation  and  operating  costs. 

Since  inspectors  have  to  move  to  different  areas  on  the  aircraft  during  a  single  task  and  all  areas  may  not 
be  accessible  to  generalized  lighting  from  a  static  source,  generalized  lighting  may  be  augmented  from 
a  combination  of  static  portable  sources,  and  then  further  augmented,  if  necessary,  using  flashlights. 

It  is  proposed  to  use  the  Task  Analyses  performed  so  far  and  lighting  surveys  of  the  inspection  work  areas 
to  determine  the  design  requirements  for  lighting  in  detail.  The  market  will  then  be  surveyed  for  available 
solutions  (e.g.,  area  lights,  flashlights,  headlights,  stand  lights)  to  choose  a  small  number  of  promising 
systems.  On-site  human  factors  evaluations  of  these  lighting  systems  will  be  performed  to  determine 
which,  if  any,  improves  visibility  of  defects  or  other  indications  to  inspectors,  while  maintaining 
portability. 

The  specific  steps  to  be  undertaken  for  this  project  are: 

1.  Site  visit  and  task  analysis  to  determine  specific  vi.sual  requirements  and  lighting  requirements  of 
tasks,  and  the  current  visual  environment.  Luminance  and  illuminance  will  be  measured  throughout 
the  hangar  to  determine  consistency  and  adequacy.  A  checklist  of  visual  factors  (from  Drury. 
1990a)  will  be  used  to  assess  the  adequacy  for  the  specific  tasks  performed. 

2.  Survey  market  for  available  solutions  to  identify  promising  systems  for  illumination,  diffusion  and 
specialized  lighting. 

3.  On-site  human  factors  evaluation  of  selected  lighting  sy.stem  to  demoastrate  advantages.  This  will 
include  performance  evaluation  (speed,  accuracy)  as  well  as  operator  acceptability  and  cost. 

4.  Produce  a  set  of  design  recommendations  which  can  be  used  as  the  basis  for  future  lighting  design. 
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5.23  FUTURE  SHORT-TERM  PROJECTS 


The  research  will  continue  to  focus  on  long-  as  well  as  short-  term  projects.  These  short-term,  immediate 
payoff  projects  will  study  such  topics  as  using  portable  computers  for  development  and  evaluation  of 
multi-level  job  cards.  Such  studies  will  involve  airline  and/or  manufacturer  participation  to  ensure  that 
research  results  and  by-products  can  readily  be  transitioned  into  aircraft  maintenance  work  environments. 

5.3  LONG-TERM  RESEARCH  PROJECTS 

From  Phase  I  came  a  wealth  of  Task  Analysis  data  and  descriptions  of  specific  inspection  and  maintenance 
organizations  at  many  carriers.  In  addition,  information  from  the  FAA/OAM  meetings,  reports  and  visits 
to  aircraft  manufacturers  and  specialized  equipment  suppliers,  gave  a  clear  description  of  the  inspection 
and  maintenance  system  to  the  human  factors  engineers  involved.  The  Phase  I  report  (Shepherd,  et  al., 
1991)  made  a  first  attempt  to  merge  this  data  with  existing  and  current  concepts  in  human  factors.  An 
obvious  need  was  to  perform  this  integration  at  a  deeper  level  to  guide  the  long-term  human  factors  needs 
of  the  aviation  maintenance  industry.  During  Phase  II,  this  step  was  undertaken;  system  demands  were 
interpreted  in  terms  of  known  human  capabilities  and  limitations. 

The  first  fruits  of  this  process  were  four  reports  which  covered  a  framework  for  human  reliability  in  this 
field,  a  detailed  examination  of  the  information  environment,  an  analysis  of  the  effects  of  time  on 
inspection  (especially  the  speed/accuracy  tradcofO.  and  a  study  of  the  improvement  of  training  for  visual 
inspection.  These  reports  are  listed  with  other  publications  at  the  end  of  this  section.  The  findings  of  each 
report  are  summarized  in  Sections  5.3.1  through  5.3.5,  augmented  where  necessary  by  off-line 
experiments.  Additionally,  a  joint  venture  between  the  FAA  and  the  CAA  on  in.spection  is  presented  as 
Section  53.6. 

5.3.1  COMPUTER-BASED  INSPECTION  EXPERIMENTS 

It  became  apparent  that  the  traditional  experimental  work  in  aviation  inspection  was  not  always  the  best 
way  to  perform  human  factors  evaluations.  Studies  of  crack  detection  probabilities  (reO  have  been  targe, 
costly,  and  complex,  but  have  not  addressed  many  of  the  human  factors  issues  beyond  the  psychophysics 
of  NDI  equipment.  Factors  such  as  training  method,  information  environment,  and  lime  pressure  have  not 
been  systematically  considered.  Thus,  the  need  was  recognized  for  a  low-cost  but  realistic  simulator  for 
aircraft  inspection.  Its  purpose  is  not  to  provide  a  point  estimate  of  the  probability  of  detection  of  a  given 
crack,  but  rather  to  determine  how  inspection  performance  is  affected  by  manipulable  human  factors  such 
as  those  above.  There  is  sufficient  knowledge  of  models  of  human  inspectors  (e.g.,  review  by  Drury, 
1991)  to  be  able  to  determine  which  aspects  of  the  real  task  to  retain  if  a  simulator  is  to  be  "realistic". 

Two  simulation  programs  were  implemented  on  a  SUN  Sparc  station  1  workstation,  one  for  an  NDI  task 
(eddy  current  inspection  of  rivets)  and  the  other  for  a  visual  task  (visual  inspection  of  rivets  and  sheet 
metal).  These  programs  are  discussed  below. 
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5J.1.1  NDI  (Eddy-Current)  Inspection  Program 

The  inspection  task  consists  of 
inspecting  rows  of  fuselage  rivets  for 
cracks  using  an  eddy-current  probe. 

The  simulator  display  consists  of  four 
windows  (Figure  5.1)  as  follows: 


Inspection  Window.  This  window 
displays  the  rivets  to  be  inspected.  Six 
rivets  pet  row  are  displayed  at  a  time. 

The  simulation  program  has  the 
capability  to  display  multiple  rivet 
rows  at  a  time.  During  the  training 
session  a  circle  is  placed  around  each 
rivet  to  help  the  subjects  in  defining 
the  optimal  probe  path  around  the  rivet 
for  defect  detection.  On  the  upper 
right  hand  comer  of  this  window  there 
is  an  indicator  that  is  green  when  the 
subject  is  in  the  inspection  mode. 

During  this  mode,  the  subject  is  able  to  inspect  and  classify  (defective/non-defective)  the  rivets,  but  has 
no  access  to  any  of  the  functions  outside  the  current  window.  To  obtain  access  to  these  functions,  the 
subject  has  to  click  the  left  mouse  button  near  any  of  the  rivets.  This  results  in  a  circular  marker  being 
placed  around  that  rivet  and  the  inspection  indicator  light  turns  white,  indicating  the  inspection  mode  is 
switched  off. 

Macro-View  and  Directionals.  The  macro-view  in  the  upper  left  window  allows  the  subject  to  have  a 
view  of  the  total  inspection  area  and  its  relation  to  the  aircraft  fuselage.  Thus,  for  a  400  rivet  inspection 
task,  while  only  six  rivets  are  seen  in  the  inspection  window,  the  entire  400  rivets  are  marked  (on  a 
smaller  scale)  in  the  macro-view.  A  click  on  the  where-am-I  button  places  a  circle  around  the  area  of  the 
macro-view  currently  in  the  inspection  window.  Thus,  the  subject  is  able  to  determine  where  he/she  is 
at  any  point  in  time  with  relation  to  the  enure  task. 

The  direcUonals  consists  of  four  square  areas  marked  left,  up,  right  and  down  (LAJ/R/D,  clockwise). 
Clicking  the  left  mouse  button  on  any  one  of  these  areas  shifts  the  view  (scrolls)  in  the  inspection  window 
in  the  indicated  direction. 

Eddy  Current  Meter.  The  defect  indication  is  displayed  on  the  meter  indicator  in  the  upper  right  window 
of  the  monitor  screen.  The  meter  has  a  fixed  scale  with  divisions  marked  from  0  to  100,  and  a  moving 
indicator.  A  red  marker  is  provided  that  can  be  set  by  the  subject  at  any  point  on  the  scale.  The 
deflection  of  the  needle  (from  its  resting  position  at  zero)  beyond  this  set  point  (default  =  60)  produces 
an  auditory  alarm  as  well  as  a  red  na.sh  of  the  indicator  light  at  the  apex  of  the  meter. 
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The  point  of  the  needle  is  deflected  if  any  of  the  following  happen: 

1.  The  mouse  cursor  is  moved  over  a  crack  on  the  rivet  (the  cracks  themselves  are  not  visible). 

2.  The  mouse  cursor  is  moved  over  a  grey  spot  (indicating  corrosion,  or  dent;  randomly  placed  across 
rivets). 

3.  The  mouse  cursor  is  very  close  to,  or  moved  over,  the  rivet  head  itself. 

Subjects  are  instructed  that  if  the  deflection  is  greater  than  60%  and  they  Judge  it  to  be  from  a  crack,  then 
the  rivet  should  be  marked  bad. 

Lower  Right  Window.  This  area  contains  functional  (dialogue)  buttons.  Activation  of  the  zoom  button 
allows  the  subject  to  take  a  closer  look  at  the  current  rivet  to  be  inspected.  The  zoom  is  incremental  and 
magnifies  the  area  to  twice  its  original  size  (within  the  inspection  window)  at  every  click.  A  mouse  click 
on  the  unzoom  area  restores  the  inspection  window  to  its  original  condition.  Clicking  on  the  "break"  area 
stops  all  clocks  and  covers  the  inspection  window  to  allow  the  subject  to  take  breaks.  Clicking  on  the 
"clock"  area  displays  the  time  elapsed  in  the  task.  The  other  functional  buttons  includes  "display  non¬ 
routine  card,"  "display  workcard,"  and  "turn  rivet  numbers  on/off." 

The  p'ogram  also  has  the  facility  for  recording  the  subject’s  assessment  of  workload  using  the  Pearson 
Feeling  Tone  Checklist  and  the  Modified  Cooper-Harper  Scale.  These  two  scales  appear  for  response  at 
the  end  of  pre-set  intervals. 

5.3.U  Visual  Inspection  Program 

To  simulate  visual  inspection,  the  SUN  Sparc  station  1  is  used  with  a  program  having  similar  logic  and 
displays  to  the  NDI  program.  The  major  differences  are  that  detection  is  visual,  and  that  the  eddy-current 
meter  is  obviously  absent.  In  this  task  the  inspector  searches  for  multiple  defect  types  and  classifies  them 
into  different  severity  categories.  The  various  fault  types  with  their  descriptions  are: 

1.  Missing  Rivet:  A  rivet  missing  from  the  rivet  hole. 

2.  Damaged  Rivet:  Part  or  all  of  'he  rivet  head  is  damaged  resulting  in  jagged  edges. 

3.  Pooched/Dished  Rivets:  Rivets  with  a  center  which  appears  raised  or  sunken. 

4.  Loose  Rivets:  Rivets  running  loose  in  the  rivet  holes. 

5.  Rivet  Cracks:  Cracks  which  originate  at  the  edges  of  the  rivets  and  propagate  upwards  and 

outwards. 

6.  Dents:  Sheet  metal  damage  in  the  aircraft  fuselage  represented  by  sunken  areas. 

7.  Corrosion:  Damage  to  sheet  metal  surface  represented  by  patches  of  discolored  or  raised  skin. 

Depending  upon  the  .severity  of  the  delect  type,  the  defects  can  be  cla-ssilied  into  critical  and  noncritical 
defects. 

The  layout  of  the  multi-window  simulated  inspection  task  is  .shown  in  Figure  5,2.  The  function  of  each 
window  is  as  follows: 

Inspection  Window.  The  area  currently  being  inspected  is  shown  in  the  left  (large)  window.  To  .simulate 
the  use  of  local  lighting,  such  as  a  flashlight  beam,  only  a  smaller  window  within  this  area  is  fully 


74 


Human  Reliability  in  Aircraft  Inspection 


illuminated.  Within  this  smaller 
window,  faults  can  be  seen  and 
responded  to  by  clicking  them  using 
the  mouse  button.  The  entire  area  of 
tlie  inspection  window  can  be  viewed 
by  successive  movements  of  the 
smaller  illuminated  window. 

Search  Monitor  Window.  This  is  a 
monitoring  device  which  helps  the 
inspector  keep  track  of  the  window 
movement  in  the  inspection  window. 

It  provides  the  inspector  feedback  as  to 
the: 

1.  Point  of  previous  fixation 

2.  The  sequence  (pattern)  adopted 

by  the  inspector  Figure  5.2  Layout  of  the  Simulated  Inspection  Task 

3.  The  area  covered  (viewed 

through  the  window)  up  to  the  current  time. 

The  small  illuminated  window  in  the  inspection  window  is  represented  by  a  tile  in  the  search  monitoring 
window.  As  the  window  is  moved,  so  does  the  tile.  The  tile  has  a  different  color  from  its  illuminated 
background  area.  The  background  color  changes  to  the  color  of  the  tile  as  the  tile  passes  over  it, 
indicating  that  the  corresponding  areas  have  been  fixated  (covered  by  the  window).  The  darkest  shade 
of  the  tile  is  the  point  of  previous  fixation.  The  sequence  is  given  by  the  shade  of  the  color-lighter 
shades  indicate  earlier  fixations  in  sequence  while  darker  shades  indicate  later  fixations. 

Macro  View  Window.  This  window  represents  the  entire  task  to  be  inspected,  and  can  be  looked  upon 
as  the  global  coordinate  referencing  system.  Thus,  it  provides  information  to  the  inspector  as  to  his 
current  position  with  reference  to  the  entire  task. 

5.3.2  A  FRAMEWORK  FOR  HUMAN  RELIABILITY  IN  AIRCRAFT  INSPECTION 

Maintaining  civil  aircraft  worthiness  requires  the  reliability  of  a  complex,  socio-technical  system.  This 
system’s  reliability  i.,  dependent  on  the  reliability  of  its  components  (i.e..  equipment,  inspectors,  the 
physical  environment),  and  on  how'  reliably  these  components  interact.  Most  errors  in  aircraft  inspection 
and  maintenance  can  ultimately  be  attributed,  at  some  level,  to  a  human-system  mismatch.  Operators  may 
cause  errors  outright,  or  more  likely,  human  information  processing  limitations  and  characteristics  may 
be  "catalytic"  factors  (Rou.se  and  Rou.se.  19H3),  combining  with  other  component  characters  to  evolve 
"sneak  paths"  (Rasmussen,  1982)  to  error  situatii>ns. 

The  asse.ssment  of  human  error  in  complex  systems  is  currently  undergoing  .somewhat  of  a  renai.ssance 
(Brown  and  Groeger,  1990).  Classification  schemes  of  errors  have  expanded  from  the  early 
"omi.ssion/commi.ssion"  cla.ssification  (.Swain  and  (iutlman,  1983  and  Meisier,  1971)  to  more  behavior- 
ba.sed  cla.s.sifications  (e.g.,  Norman,  1981;  Rasniu.ssen,  1982;  Rouse  and  Rouse.  1983,  and  Rea,s(m.  1990). 
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While  error  classifications  based  on  task  characteristics  may  provide  a  convenient  descriptive  format  for 
errors,  error  models  based  on  human  behavior  can  define  causal  mechanisms  of  errors.  Identification  of 
causal  mechanisms  and  catalytic  factors  is  necessary  for  predicting  errors  and  thereby  designing  error 
tolerant  systems.  The  approach  taken  here  is  to  use  a  behavior-based  and  system-based  human  error 
classification  scheme  to  identify,  predict,  prevent  or  reduce,  and  report  errors  in  aircraft  inspection  and 
maintenance. 

This  section  focuses  on  describing  a  methodology  to  accomplish  these  goals.  Section  5.3.2  provides  more 
detail  in  defining  information  flow,  and  deriving  information  requirements  which  will  prevent  or  mitigate 
the  effects  of  information  flow-related  errors.  Both  this  section  and  Section  5J.3  are  responses  to  FAA 
project  activities  and  exist  elsewhere  as  two  self-contained  separate  reports  (Latorella  and  Drury,  1991, 
and  Drury  and  Prabhu,  1991,  respectively).  These  reports  have  been  considerably  abbreviated  for  their 
presentation  in  this  report.  Both  efforts  use  Rasmussen's  (1986)  cognitive  control  levels  and  Rasmussen 
and  Vicente’s  (1989)  systemic  error  mechanisms  extensively  as  a  conceptual  foundation.  Both  efforts  also 
begin  with  Drury’s  (1991)  Failure  Modes  and  Effects  Analysis  (FMEA)  of  errors  in  aircraft  inspection  and 
maintenance.  These  concepts  and  the  FMEA  are  presented  only  in  the  first  section  to  avoid  redundancy. 
As  a  result,  some  of  the  material  presented  in  Section  5.3.3  is  dependent  on  the  theoretical  and  data 
analysis  foundation  described  in  this  section. 

5.3J.1  Approaches  to  Human  Error 

5.3.2.1.1  Quantitative  Approaches 

Early  efforts  to  incorporate  human  performance  in  the  evaluation  of  system  reliability  spawned  the  field 
of  Human  Reliability  Assessment  (HRA).  These  methods  attempt  to  assess  human  reliability  with  the 
same  techniques  used  to  assess  equipment  reliability  (Meister,  1971).  They  seek  to:  (1)  develop  extensive 
databases  of  human  reliability  data  for  elemental  tasks,  (2)  provide  a  method  for  combining  these  estimates 
to  generate  a  measure  of  human  reliability  within  the  system,  (3)  use  this  measure  of  human  reliability 
directly,  as  the  reliability  of  the  human  as  a  system  component,  in  evaluations  of  total  system  reliability 
by  Probabilistic  Risk  Assessment  (PRA).  Early  HRA  methods  are  criticized  for  their  overly-structured, 
and  hence  cumbersome,  representations  of  the  human’s  involvement  in  systems.  HRA  methods  are  also 
criticized  for  their  inability  to  adequately  represent  the  behavioral  mechanisms  of  human  errors  and  hence 
for  their  inability  to  prescribe,  rather  than  merely  describe,  systems  in  terms  of  their  propensity  for  error 
situations.  Quantitative  human  error  assessment  techniques  include  decompositional  probabilistic  methods 
(e.g..  Fault  Trees,  Event  Trees,  Failure  Modes  and  Effects  Analysis),  classical  reliability  theory  based  on 
Markov  modeling,  stochastic  simulation  modeling,  and  a  variety  of  other  techniques  (e.g.,  HCR,  TESEO, 
SLIM-MAUD).  Thc.se  approaches  are  each  de.scribed  and  critiqued  in  Latorella  and  Drury  (1991).  Lock 
and  Strutt  (198.5)  have  investigated  quantitative  human  error  modeling  in  the  aircraft  inspection  and 
maintenance  context. 

5.3.2. 1.2  Qualitative  Approaches 

Several  researchers  have  arrived  at  behavior-based  classification  schemes  for  human  errors.  Tlio.se  of 
Norman,  Hollnagel,  Rasmussen,  and  R()u.se  and  Rouse  are  described  below.  Elements  of  these  schemes 
have  been  in  approaches  to  managing  errors  in  aircraft  maintenance  and  inspection. 
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Norman.  Norman  (1980, 1981)  classifies  human  error  into  two  fundamental  categories:  slips  and  mistakes. 
Slips  result  from  automated  behavior  when  the  intention,  the  goal,  is  correct  but  some  aspect  of  the 
execution  is  flawed.  Mistakes,  in  contrast,  are  the  result  of  flawed  cognitive  processes,  such  as  formation 
of  the  wrong  goal.  Slips  are  usually  minor  errors  and  are  often  evident  and  corrected  by  the  perpetrator. 
Mistakes,  however,  are  more  serious  errors,  and  are  sometimes  opaque  to  the  perpetrator.  Mistakes  are 
therefore  usually  difficult  to  observe  and  recover.  Slips  are  partially  due  to  limitations  in  attention  and 
therefore  are  more  likely  to  occur  in  distracting,  time-sharing,  boring,  or  stressful  situations.  Norman 
identifies  six  types  of  slips:  capture  errors,  description  errors,  data  driven  errors,  associative  activation 
errors,  loss  of  activation  errors,  and  mode  errors.  Descriptions  of  these  types  of  slips  and  examples  related 
to  aircraft  inspection  and  maintenance  can  be  found  in  the  original  report  (Latorella  and  Drury,  1991). 
Norman’s  (1981)  classification  is  intuitively  appealing  and  useful  for  describing  errors.  However,  the 
slip/mistake  classification  is  not  detailed  enough  to  describe  what  specific  aspects  of  human  information 
processing  generate  errors. 

Hollnagel.  Hollnagel  (1989)  introduces  the  conceptual  distinction  between  error  phenotypes  and  error 
genotypes.  Error  phenotypes  are  observable  states  which  are  deemed  undesirable.  Error  genotypes  are 
the  generative  mechanisms  of  these  observable  states.  Error  phenotypes  are  manifestations  of  error 
genotypes  expressed  in  a  particular  environment.  While  Hollnagel  allows  that  combining  genotypes  and 
phenotypes  provides  a  more  complete  psychological  description  of  human  error,  he  holds  an  empiricist’s 
view  for  the  purpose  of  system  design:  in  order  to  automate  error  detection,  errors  can  only  be  expressed 
in  terms  of  phenotypes.  He  therefore  proposes  a  taxonomy  to  operationalize  phenotypes,  describe  complex 
phenotypes  (combinations  of  simple  phenotypes),  and  to  provide  a  basis  for  a  computer  program  which 
detects  error  situations.  Hollnagel’s  distinction  between  error  phenotypes  and  error  genotypes  is  important 
and  is  used  in  the  development  of  this  paper’s  approach  to  managing  aircraft  inspection  and  maintenance 
errors. 

Rasmussen.  Rasmussen  has  contributed  to  HRA  in  two  veins:  he  has  developed  models  of  human 
performance  in  an  effort  to  identify  fundamental  causes  of  human  error,  and  he  has  related  and  defined 
the  importance  of  qualitative  human  error  modeling  to  system  reliability.  Rasmussen  departs  from  the 
more  traditional  approaches  in  his  conceptualization  of  human  error.  He  does  not  rely  on  the  constrained 
definition  of  human  error  presented  in  most  HRA  techniques,  rather  he  states  that  what  is  human  error  is 
defined  by  not  only  the  human,  but  by  system  and  operational  tolerances  (Rasmussen,  1982).  Rasmussen 
also  argues  that  human  errors  defined  by  the  outcome  of  events  should  not  necessarily  be  attributed  to  a 
human  having  performed  incorrectly.  For  example,  should  an  error  resulting  from  a  new  situation  be 
attributed  to  the  human?  If  an  error  provides  feedback  about  the  system  without  compromising  system 
functioning,  should  it  still  be  considered  something  to  avoid?  Rasmussen  also  defines  stipulations  for 
collecting  HRA  error  rate  probabilities  and  states  the  case  for  qualitative  error  modeling  to  aid  HRA  in 
ways  that  error  rates  can  not,  such  as  prediction  and  corrections  of  errors,  especially  of  low  probability, 
high  impact  "sneak  paths".  Rasmussen  (1982)  developed  a  cla.ssification  of  human  error  towards  this  end. 

The  skill-rule-knowledge  (SRK)  framework  proposed  by  Rasmu.ssen  (1986)  classifies  human  behavior  into 
three  categories  of  ascending  complexity:  skill-ba.scd  behavior,  rule-ba.sed  behavior  and  knowledge-ba.sed 
behavior.  Any  decision  is  made  at  the  lowe.st  level  possible,  with  progression  to  higher  levels  only  when 
a  lower  level  fails  to  reach  a  decision. 
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Skill-based  behavior  represents  psychomotor  behavior  without  conscious  control,  consisting  of  automated 
routines  that  are  driven  by  sensory  data  received  as  "signals"  from  the  environment  (Rasmussen,  1986). 
Signals  represent  information  that  is  a  quantitative  indicator  of  the  temporal  and  spatial  aspxjcts  of  the 
environment,  and  may  trigger  skill-based  behavior  by  activating  the  automated  behavioral  routines  of  the 
human.  Skill-based  behavior  is  normally  based  on  feedforward  control  and  proceeds  without  conscious 
attention. 

From  the  aircraft  inspection  viewpoint,  the  movement  of  the  pencil  probe  around  a  rivet  or  a  sliding  probe 
along  a  stringer  (a  row  of  rivets)  during,  for  example,  an  eddy  current  inspection  or  an  ultra-sonic 
inspection,  represents  skill-based  sensorimotor  performance  involving  some  amount  of  feedback  control. 
Similarly,  the  pre-attentive  phase  of  visual  search,  as  well  as  the  extra-foveal  process  in  extended  visual 
search  can  be  considered  to  be  skill-based  behaviors  that  are  data  driven  and  based  on  feedforward  control. 

Rule-based  behavior  represents  consciously  controlled,  goal-oriented  behavior  guided  by  rules  or 
procedures  for  action.  These  rules  are  stored  patterns  of  behavior  that  have  been  empirically  derived 
during  previous  occasions  or  communicated  as  instructions  from  an  external  source  (Rasmussen,  1986). 
Information  during  rule-based  performance  is  perceived  as  "signs"  which  represent  information  that 
activates  or  modifies  the  rules  and  depicts  situations  or  environmental  features  along  with  the  conditions 
to  act  (Rasmussen,  1986).  Rule-based  behavior  proceeds  towards  a  goal,  utilizing  feedforward  control 
through  rules  and  without  demanding  any  deeper  reasoning  on  the  part  of  the  human. 

In  aircraft  inspection,  an  experienced  inspector  interpreting  the  deflection  of  the  ultra-sonic  meter,  or  the 
pattern  traced  on  an  oscilloscope  during  eddy  current  testing,  can  be  assumed  to  be  indulging  in  a 
rule-based  behavior  if  the  "signs"  are  familiar.  Similarly,  the  extra-foveal  process  in  search  where  cues 
on  the  periphery  guide  the  next  fixation  can  be  considered  a  rule-based  behavior.  Rule-based  search  can 
also  result  from  information  gathered  in  the  foveal  component,  for  example  bulging  of  aircraft  skin  triggers 
search  for  corrosion.  Pre-determined  search  strategies,  as  a  result  of  past  experience,  training,  or  work 
card  instructions,  can  also  lead  to  a  rule-ba-sed  behavior. 

Knowledge-based  behavior  repre.scnts  goal -control  led,  problem-solving  performance  in  unfamiliar 
situations.  It  requires  a  functional  understanding  of  the  system,  analysis  of  Uie  current  state,  and  response 
of  the  environment  based  on  conscious,  advanced  reasoning  while  utilizing  feedback  control  for  error 
correction  (Rasmussen,  1986).  During  knowledge-ba.sed  behavior,  the  human  perceives  information  as 
"symbols",  i.e.,  concepts  about  the  functional  aspects  of  the  environment  which  refer  to  an  internal 
representation  that  can  be  used  by  the  human  for  reasoning  (Rasmussen,  1986). 

In  aircraft  inspection,  knowledge-ba.sed  behavior  can  occur  in  NDl,  for  example  during  eddy  current 
testing  of  rivets,  when  the  in.spector  sees  a  curve  traced  on  tfie  oscilloscope  screen  of  a  shape  never 
encountered  before.  In  this  case  the  inspector  has  to  use  the  knowledge  of  eddy  current  technology, 
knowledge  about  the  instrument,  knowledge  about  the  aircraft,  etc.,  to  interpret  whether  the  signal 
represents  a  crack  or  not.  Along  similar  lines,  the  u.se  of  cues  to  delect  visual  defects  needs  active 
rea,soning  (knowledge-based  behavior)  until  the  a.s.sociaiion  of  the  cue  to  the  defect  is  ct)nnrmed,  in  which 
case  the  cue  will  trigger  rule-ba.sed  behavior. 
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Rasmussen  (1982)  provides  a  framework  for  classifying  causes  of  human  error  as  a  function  of  situational 
and  task  characteristics  and  the  error  phenomenon.  Basic  error  mechanisms  are  derived  through  the  use 
of  a  human  information  processing  model,  linking  human  decision-making  and  responses  to  internal 
processes.  His  model  can  be  used  to  describe  human  behavior  over  the  three  levels  of  cognitive  control, 
and  can  be  used  to  indicate  decision  aiding  devices  and  training  needs  at  these  different  levels.  He 
specifically  mentions  that  systems  must  be  designed  with  interlocks  and  barriers  where  it  is  unreasonable 
to  expect  operators  not  to  err  and  that  systems  should  allow  errors  to  be  observed  and  reversed.  A  related 
work  (Rasmussen  and  Vicente,  1989)  identifies  four  systemic  error  mechanism  categories:  (1)  effects  of 
learning  and  adaptation.  (2)  interference  among  competing  control  structures,  (3)  lack  of  resources,  and 
(4)  stochastic  variability  of  individuals.  Rasmussen  and  Vicente  (1989)  describe  examples  of  errors  within 
these  categories  and  cognitive  control  levels  (see  Table  5,5).  Similarly,  Drury  and  Prabhu  (1991)  used 
the  cognitive  control  classification  to  organize  error  shaping  factors  (see  Table  5.6). 


EFFECTS  OF  LEARNING  AND  ADAPTATION: 

•  Knowledge-based: 

•  Rule-based: 

•  Skill-based: 

Search  for  information  and  hypothesis  testings  in  novel  situations  may  lead  to  acts  which  are 
judged  as  errors  after  the  fact; 

The  law  of  least  effort  may  lead  to  underspecified  cues; 

Optimization  of  motor  skill  needs  feedback  from  boundaries  of  acceptable  performance  (speed- 
accuracy  tradeoff): 

INTERFERENCE  AMONG 

COMPETING  CONTROL  STRUCTURES: 

•  Knowledge-based: 

•  Rule-based: 

•  Skill-based: 

False  analogies;  interference  in  means-end  hierarchy: 

Functional  fixation:  adherence  to  familiar  rules: 

Capture  by  frequently  used  motor  schemata: 

LACK  OF  RESOURCES; 

•  Knowledge-based: 

•  Rule-based: 

•  Skill-based: 

Limitations  of  linear  reasoning  in  causal  networks:  insufficient  knowledge,  time,  force,  etc.; 
Inadequate  memory  for  rules; 

Lack  of  speed,  precision,  force; 

STOCHASTIC  VARIABIUTY; 

•  Knowledge-based: 

•  Rule-based: 

•  Skill-based: 

Slips  of  memory  in  mental  models; 

Erroneous  recall  of  data  or  oarameters  related  to  rules; 

Variability  of  attention:  variability  of  motor  parameters,  motor  noise  (variation  in  force,  precision 
of  movements); 

Table  5.5  Potential  Errors  De.scribed  by  Level  of  Cognitive  Control  and  Systemic  Error  Mechanisms 
(Rasmussen  and  Vicente,  1989) 


Rouse  and  Rouse.  Rou.se  and  Rouse  (1983)  propo.se  a  behavioral  classification  scheme  for  human  errors 
which  borrows  heavily  from  Rasmus.sen’s  contributions.  Tliey  attempt  to  analyze  human  error  in  terms 
of  cau.scs,  as  well  as  contributing  factors  and  events.  Tlieir  scheme  organizes  human  errors  around 
Rasmu.ssen’s  flow  model  (1976)  of  an  operator’s  information  proce.ssing  task.  Tliis  model  gives  the 
following  .steps  in  task  performance: 
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KNOWLEDGE-BASED  ERROR  SHAPING  FACTORS 

Information  Overload 

•  Excessive  demands  on  memory. 

Incomplete  Knowledge 

•  Knowledge  lacking  on  system  dynamics,  parameters,  side  effects,  etc. 

Delayed  Feedback 

•  Delays  in  feedback  concerning  the  execution  of  decisions. 

Bounded  Rationality 

•  Limited  cognitive  capacity  relative  to  problem  size  leads  to  satisficing 
behavior. 

Memory  Cueing 

•  Familiar  information  in  LTM  is  cued  by  problem  content. 

Insufficient  Consideration  of  Process 
Causal  Series  Vs.  Nets 

•  Past  history  disregarded  v^en  handling  dynamic  current  events. 

Attentional  Limitations 

•  Oversimplification  of  causality,  thinking  in  terms  of  immediate  goals. 

Confirmation  Bias 

•  Finite  resources  of  attentional  processes. 

Vagabonding 

•  Tendency  to  maintain  current  hypothesis  facing  contradictory  evidence. 

Overconfidence 

•  Moving  amongst  issues  without  detailed  consideration  of  any  one. 

Memory  Slip 

•  Excessive  belief  in  ti\e  correctness  of  one’s  own  knowledge. 

Selectivity 

•  Memory  failure  due  to  faulty  activation  of  schemata. 

•  Selectively  process  infomnation.  Attention  to  wrong  features  of  task  can 

Biased  Reviewing 

result. 

Illusory  Correlation 

•  Error  in  reviewing  planned  course  of  action. 

Lack  of  Resources 

•  Failure  to  detect  correlation  or  understand  the  logic  of  covariation. 

Complexity  Problems 

•  Lack  of  mental  capacity  for  causal  reasoning,  insufficient  knowledge. 

•  Problems  in  understanding  system  due  to  its  complex  nature. 

RULE-BASED  ERROR  SHAPING  FACTORS 

Availability 

•  Tendency  to  use  intuitive  oiles  or  use  rules  that  readily  come  to 

mind. 

Countersigns 

•  These  are  input  indications  that  the  more  general  rule  is 

inapplicable. 

Rigidity 

•  Mindset  (cognitive  conservatism)  results  in  refusal  to  change 

familiar  procedure. 

Encoding  Deficiency 

•  Encode  inaccurately  or  fail  to  encode  the  properties  of  the  problem 

space. 

Inadvisable  Rules 

•  Rules  that  satisfy  immediate  goals  but  can  cause  errors  due  to  side 

effects. 

Wrong  Rules 

•  Rules  that  are  wrong  for  the  current  situation. 

Inelegant  Rules 

•  Rules  that  achieve  the  goal  but  are  inefficient. 

First  Exceptions 

•  Errors  caused  on  first  occasion  that  is  an  exception  to  the  general 

njle. 

SKILL-BASED  ERROR  SHAPING  FACTORS 

Omissions 

•  Omission  of  actions/action  sequences  needed  to  achieve  a  specified 

goal. 

Perceptual  Confusion 

•  A  familiar  match  is  accepted  instead  of  the  correct  match. 

SATO 

•  Speed  accuracy  tradeoff. 

Motor  Schemata  Capture 

•  Focussed  attention  absence  leads  to  takeover  by  frequently  used 

schemata. 

Stochastic  Variability 

•  Variability  in  control  of  movements. 

Reduced  Intentionatity 

•  Delay  between  intention  and  action. 

Repetitions 

•  Actions  that  are  unnecessarily  repeated. 

Reversals 

•  Unintentionally  reversing  an  action  just  committed. 

Interference 

•  Potential  problems  stemming  from  concurrent  activities. 

Table  5.6 


Defimlions  of  Error  Shaping  Factors  (Erabhu,  Sharit  and  Drury,  1992) 
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1.  Observation  of  system  state 

2.  Choice  of  hypothesis 

3.  Testing  of  hypothesis 

4.  Choice  of  goal 

5.  Choice  of  procedure 

6.  Execution  of  procedure. 

This  classification  scheme  has  been  used  to  record  and  analyze  human  errors  in  several  contexts:  (1) 
detection,  diagnosis,  and  compensation  of  engine  room  failures  in  a  supertanker  (van  Eckhout  and  Rouse, 
1981),  (2)  human  errors  in  troubleshooting  live  aircraft  power  plants  (Johnson  and  Rouse,  1982),  and  (3) 
aircraft  pilots  in  mission  flights  (Rouse,  Rouse  and  Hammer,  1982).  Results  of  these  studies  have  been 
applied  to  the  improvement  of  training  programs  and  the  development  of  checklists  and  other  decision 
aids. 

5.3J.1.3  Human  Error  in  Aircraft  Inspection  and  Maintenance 

Whereas  previous  research  in  aircraft  inspection  and  maintenance  has  utilized  various  empirical  human 
factors  techniques,  this  effort  uses  a  behavior-based  human  error  modeling  approach,  housed  in  a 
conceptual  aircraft  inspection  and  maintenance  system  model  (see  Figure  5.3).  The  system  model 
provides  a  framework  for  error  classification  and  therefore,  a  basis  for  improved  error  management.  The 
following  section  describes  the  system  model  of  aircraft  inspection  and  maintenance.  The  final  section 
details  how  the  model  might  be  useful  for  managing  aircraft  inspection  and  maintenance  errors. 

5.3.2J  A  System  Model  for  Human  Error  in  Maintenance  and  Inspection 

The  fact  that  errors  emerge  from,  and  are  defined  by,  the  interaction  of  system  characteristics,  indicates 
the  necessity  of  a  system  approach  to  the  description  and  control  of  these  errors.  Such  a  system  view  of 
aircraft  inspection  and  maintenance  includes  not  only  the  traditional  interaction  of  the  operator  and  task 
requirements,  but  also  includes  operator  interactions  with  equipment,  documentation,  and  other  personnel 
within  the  constraints  imposed  by  the  environment.  The  system  model  (Latorella  and  Drury,  1991) 
contains  four  components:  operators  (personnel),  equipment,  documentation,  and  task  requirements.  These 
components  are  subject  to  consftaints  of  both  the  physical  environment  and  the  social  environment.  The 
job  component  can  also  be  considered  as  a  subset  of  the  organizational  environment  in  which  tasks  are 
defined.  Similarly,  the  workspace  component  is  a  subset  of  the  physical  environment.  This  conceptual 
model  is  two  dimensional  as  shown  in  Figure  53.  The  temporal  sequence  of  the  individual  tasks  defines 
an  axis  orthogonal  to  the  page.  All  other  system  elements  interact  with  the  current  task  component  as 
shown  in  the  plan  view.  Each  individual  task  is  subject  to  different  combinations  and  degrees  of 
influences  from  other  system  components,  presented  below. 

Operators.  Aircraft  maintenance  and  inspection  operators  (O)  differ  between  organizations  but  belong 
in  the  same  basic  categories:  inspectors  (perhaps  distinguished  as  either  visual  or  NDT),  maintenance, 
utility,  lead  inspectors,  lead  maintenance,  inspection  foremen,  maintenance  foremen,  production  foremen, 
and  engineers.  In  addition  to  carrying  out  sequences  of  activities,  personnel  .serve  as  informational 
resources  to  each  other.  Communication  between  personnel  can  be  viewed  as  an  information  processing 
task  similar  to  referencing  a  document.  The  organizational  structure  of  the  system  impo.scs  constraints 
on  the  amount  of,  format  of,  and  the  personnel  likely  to  engage  in,  collaborative  problem-solving 

81 


Chapter  Five 


communicaiions.  The  affective  and 
physical  characteristics  of  people  are 
also  important.  An  individual’s  affect 
can  influence  motivation  and  hence, 
performance.  Physical  characteristics 
affect  perception  (e.g.,  visual  acuity), 
access  (e.g.,  anthropometry),  and  other 
tasks. 


Equipment.  Both  visual  and  NDT 
inspection  use  equipment  (E).  There 
is  specialized  equipment  for  different 
types  of  NDT,  including:  eddy 
current,  ultrasonic,  magnetic  resonance, 

X-Ray,  and  dye  penetrant.  Visual 
inspection  requires  flashlights,  mirrors, 
and  rulers.  Use  of  this  equipment 
requires  specialized  knowledge  of  its 
operating  principles,  and  equally 
specialized  knowledge  for  the 
interpretation  of  its  output. 

Interpretation  of  visual  stimuli  or  NDT 
output  necessarily  requires  information 
processing  by  the  operator,  but  may  also  require  communication  with  other  personnel.  The  ability  to 
perceive  the  information  present  in  the  visual  stimuli  or  NDT  output  may  be  affected  by  environmental 
conditions,  such  as  poor  lighting.  The  ability  to  operate  NDT  equipment  properly  may  also  be  affected 
by  environmental  factors.  For  example,  some  temperature  and  humidity  combinations  make  precise 
movements  difficult.. 

Documents.  A  variety  of  documents  (D)  is  required  for  inspection  and  maintenance.  Workcards,  which 
may  include  graphics  and  references  to  more  comprehensive  standards  manuals,  specify  the  task  to  be 
performed.  Forms  (shift  turnovers,  NRRs)  are  used  to  communicate  between  personnel  and  to  document 
procedures,  while  additional  documentation  is  used  for  training  and  retraining  purposes.  The  ability  to 
communicate  effectively  through  documentation  is  based  on  many  factors.  The  fields  specified  on  forms 
dictate  the  information  and  the  structure  of  that  information.  Phy.sicai  characteristics  of  forms,  documents, 
and  graphics  affect  the  legibility  of  information  and  therefore,  impact  the  ability  to  accurately  perceive 
this  information.  Issues  of  comprehension  are  imprmant  for  understanding  the  content  of  documents. 
Issues  of  repre.sentation  are  central  to  ensuring  that  graphics  are  appropriate  and  useful. 

Task.  A  task  (Ti)  is  defined  as  the  actions  and  elements  of  one  workcard  or  similar  task  order.  Task 
characteristics  which  have  been  found  to  influence  inspection  include;  defect  probability,  physical 
characteristics  of  the  defect,  the  number  of  serial  in.spectit)ns.  feedforward  and  feedback  availability,  and 
whether  standards  are  u.sed  (Rodgers,  1983).  The.se  aspects  of  the  task  necessarily  interact  with  personnel, 
organizational,  job,  and  environmental  characteri.stics.  Personal  information  processing  biases  may  interact 
with  the  task  structure  and  present  problems  such  as  searching  in  the  wrong  area.  Tlie  definition  of  a 


Figure  5.3  System  Mode 
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defect  is  part  of  the  task  which  is  ultimately  established  by  the  organization.  An  indication,  which  implies 
a  defect,  is  defined  as  that  magnitude  which  indicates  that,  given  the  cost/benefit  tradeoff  of  repairing 
versus  not  repairing,  a  repair  should  be  performed.  The  organization  also  dictates  whether  feedforward, 
feedback,  and  standards  are  used  in  inspection.  The  interaction  of  task  characteristics  and  job 
characteristics  may  produce  effects  on  inspection  performance.  The  probability  of  defects  affects  the 
arousal  level  of  an  inspection  and  the  expectation  of  finding  a  fault,  which  is  also  affected  by  the  length 
of  time  an  inspector  performs  a  task  and  by  physical  factors  such  as  fatigue. 

Job.  Jobs  (J)  are  defined  by  the  collection  of  tasks  that  an  individual  is  expected  to  perform.  However, 
there  are  many  characteristics  of  the  job  which  can  not  be  described  by  the  characteristics  of  its  individual 
tasks.  Job  factors  are  derivative  of  the  organizational  environment  and  provide  constraints  for  tasks  (e.g., 
shift  durations,  work/resi  cycles,  day/night  shifts,  job  rotation  policies).  These  can  further  impact 
personnel  physical  (e.g.,  fatigue,  eyestrain),  affective  (e.g.,  motivation,  job  satisfaction),  and  information 
processing  (e.g.,  attention  allocation)  characteristics. 

Workspace.  The  workspace,  a  subset  of  the  physical  environment,  contains  the  task  and  the  equipment, 
documentation,  and  personnel  required  to  perform  the  task.  While  illumination  is  an  attribute  of  the 
physical  environment  in  general,  task  lighting  (such  as  a  llashlight)  is  an  attribute  of  the  workspace.  The 
degree  of  physical  access  afforded  by  the  workspace  is  an  important  constraint  on  performance.  Both 
these  issues  are  currently  being  researched  under  continued  funding  on  this  contract  (Gramopadhye, 
Reynolds,  and  Drury,  1992). 

Physical  Environment.  The  physical  environment  is  described  by  several  parameters:  temperature,  noise 
level  and  type  of  noises,  lighting  level  and  light  characteristics,  and  electrical  and  chemical  sources.  While 
some  of  these  factors  can  either  enhance  or  degrade  performance,  others  indicate  potentially  hazardous 
conditions.  The  level  and  spectral  characteristics  of  lighting  affects  the  perception  of  fault  indications. 
Impulse  noises  interrupt  tasks  and  may  result  in  skipped  or  unnecessarily  repeated  procedures.  The  level 
and  frequency  characteristics  of  noise  affect  the  ability  to  communicate.  Examples  of  hazardous 
conditions  in  the  physical  environment  are  exposure  to  X-rays  emitted  during  X-ray  NDT  and  fuel  fumes 
encountered  when  inspecting  the  inside  of  a  fuel  tank. 

Organizational  Environment.  The  organizational  environment,  often  ignored  in  the  analy.ses  of 
maintenance  systems,  has  been  shown  to  be  inlluential  in  the  patterns  of  work  (Taylor,  1990)  and 
therefore,  possibly  in  the  patterns  of  errors.  Factors  which  have  been  identified  as  important  include:  the 
organization  of  work  groups  (or  conversely,  the  i.solation  of  workers),  reporting  structures,  payoff 
structures  associated  with  task  performance,  trust  within  one  class  of  personnel,  trust  between  classes  of 
personnel  and  levels  of  personnel,  .selection/placement  strategies,  and  human-machine  function  allocation 
of  control  and  responsibility.  Organizational  constraints  are  infused  into  every  level  of  the  organization. 
Regulatory  agencies  such  as  the  FAA,  JAA,  and  CAA  mandate  organizational  form  to  some  extent.  Each 
organization  has  operational  strategies  and  goals.  These  external  and  internal  goals  of  the  system,  and 
constraints  on  the  system  are  operationalized  into  changes  in  organizational  structure,  physical 
environment,  task  procedures,  job  de.scriptions,  and  personnel  (skilled  or  trained). 

Using  the  System  Model.  The  model  in  Figure  5.3  is  useful  for  depicting  the  goals  of  the  system  and 
therefore  the  functions  that  should  be  supported.  The  goals  of  the  system  are  defined  by  the  requirements 
of  the  personnel  component  in  i.solation  and  in  conjunction  with  other  system  components.  Tlie  personnel 
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component  is  primarily  described  in  terms  of  information  processing  characteristics  and  limitations.  These 
characteristics  influence  the  behavior  of  individuals  and  their  experience  with  other  system  components. 
The  functions  associated  with  the  performance  of  tasks,  use  of  equipment,  and  communication  with  co¬ 
workers  are  subject  to  error  and  are  therefore  of  primary  concern.  These  functions  are  then  considered 
within  the  constraints  of  environmental  factors  which  may  affect  error  formation  and/or  propagation. 
Drury,  Prabhu,  and  Gramopadhye  (1990)  have  compiled  a  generic  function  description  of  the  maintenance 
inspection  task  requirements  as  presented  in  Section  5.1.  The  desired  outcome  for  each  of  the  task 
functions  (Drury,  1991)  which  can  be  considered  as  the  task’s  goal  can  be  stated  and,  following  Drury 
(1991),  decomposed  into  the  steps  taken  to  accomplish  the  desired  outcome  (see  Table  5.7). 


Task  1  •  INITIATE 

1.1 

1.2 

1.3 

1.4 

1.5 

1.6 

1.7 

1.8 

Correct  instructions  written. 

Correct  equipment  procured. 

Inspector  gets  instoictions. 

Inspector  reads  instructions. 

Inspector  understands  instructions. 

Correct  equipment  available. 

Inspector  gets  equipment. 

Inspector  checks/calibrates  equipment. 

Task  2  -  ACCESS 

2.1 

Locate  area  to  inspect. 

2.2 

Area  to  inspect. 

2.3 

Access  area  to  Inspect. 

Task  3  •  SEARCH 

3.1 

Move  to  next  lobe. 

3.2 

Enhance  lobe  (e.g.  illuminate,  magnify  for  vision,  use  dye  penetrant, 
tap  for  auditory  Inspection). 

3.3 

Examine  lobe. 

3.4 

Sense  indication  in  lobe. 

3.5 

Match  indication  against  list. 

3.6 

Remember  matched  indication. 

3.7 

Remember  lobe  location. 

3.8 

Remember  access  area  location. 

3.9 

Move  to  next  access  area. 

Task  4  -  DECISION 

4.1 

Interpret  indication. 

4.2 

Access  comparison  standard. 

4.3 

Access  measuring  equipment. 

4.4 

Decide  on  if  it  is  a  fault. 

4.5 

Decide  on  action. 

4.6 

Remember  decision/action. 

Task  5  -  RESPOND 

5.1 

Mark  fault  on  aircraft. 

5.2 

Record  fault. 

5.3 

Write  repair  action. 

Task  6  -  REPAIR 

6.1 

Repair  fault. 

Task  7  -  BUY-BACK 

7.1 

initiate. 

7.2 

Access. 

7.3 

Search. 

7.4 

Decision. 

7.5 

Respond. 

Table  5.7  Detailed  Breakdown  of  Aircraft  Maintenance  and  In.spection  by  Ta.sk  Step 
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Note  that  the  use  of  equipment  has  been  included  within  these  task  descriptions  and  therefore  would  not 
be  considered  separately.  The  most  ambiguous  situations  encountered  during  aircraft  inspection  and 
maintenance  typically  result  in  an  individual  referencing  another  individual  or  a  document  for  additional 
information.  These  situations  are  underspecified  and  are  usually  unanticipated.  It  is  for  these  reasons  that 
understanding  the  communication  errors  which  may  occur  at  these  junctures  is  important.  The  type  of 
communication  of  interest  here  is  only  that  related  to  task  performance,  although  other  forms  of  casual 
communication,  not  discussed  here,  may  indicate  important  aspects  of  the  organizational  and  social 
structure  of  the  system. 

Errors  must  be  described  in  the  situational  context  in  which  they  occur  in  order  to  identify  contributing 
factors.  Table  5.8  shows  some  relevant  characteristics  of  system  components  with  which  the  individual 
may  interact  for  the  ‘initiate’  task.  Relevant  characteristics  of  each  system  component  can  be  identified 
for  observed  errors.  The  effect  of  these  factors  on  performance  has  been  suggested  in  many  studies; 
however,  the  manner  in  which  performance  is  affected,  especially  by  combinations  of  factors,  requires 
additional  empirical  investigation. 


1.0  PERSONNEL 

5.0  JOB 

1 . 1  Physiological 

5.1  Physical  Factors 

1 .2  Psychological 

5.2  Social  and  Organizational  Factors 

1 .3  Personality 

6.0  ORGANIZATIONALySOCIAL 

2.0  EQUIPMENT 

6.1  Stmeture 

2.1  Hand  Tools 

6.2  Goals 

2.2  Displays 

6.3  Trust 

2.3  Control 

6.4  Motivational  Climate/Incentives 

3.0  DOCUMENTATION 

6.5  Function  Allocation/Job  Design 

3.1  Type  of  Information  Included 

6.6  Training/Selection  Methods 

3.2  Style  (Intelligibility) 

7.0  PHYSICAL  ENVIRONMENT 

3.3  Formatting  (Visual  Clarity) 

7.1  Lighting 

3.4  Content  (Usefulness, 

7.2  Noise 

Appropriateness,  Veridical) 

7.3  Tempera.ureA/entilation 

3.5  Legibility  (Physical) 

7.4  Chemical  Hazards 

4.0  TASK 

7.5  Vibration 

4.1  Physical  Requirements 

7.6  Electrical  Shock  Hazards 

4.2  Informational  Requirements 

8.0  WORKSPACE 

4.3  Characteristics 

8.1  Proximity 

8.2  Anthropometrical  Constraints 

Table  5.8  System  Component  Influencing  Factors 


5.3.2J  Previous  Research  in  Human  Error  and  Aircraft  Inspection  and  Maintenance 

There  has  not  been  a  great  deal  of  research  on  human  error  .specifically  related  to  inspection  and 
maintenance,  less  still  targeted  to  the  inspection  and  maintenance  of  aircraft.  Three  approaches  are 
discussed  below  which  address  this  specific  research  area.  Lock  and  Strutt  (1985)  employ  a  fault  tree 
analysis  approach  to  investigating  and  quantifying  human  error  in  aircraft  inspection.  Drury  (1991) 
developed  an  error  taxonomy  of  aircraft  inspection  ba.sed  on  a  failure  modes  and  effects  analysis.  Drury 
(1991)  also  has  shown  a  classification  scheme  for  aircraft  inspection  errors  based  on  Rou.se  and  Rouse’s 
(1983)  behavioral  framework  for  investigating  errors.  The.se  contributions  are  reviewed  below. 
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Lock  and  Strutt  (1985)  begin  their  reliability  analysis  of  inspection  with  a  microstructural  model  of  the 
inspection  process.  They  use  this  model  to  "develop  a  flow  chart  (Figure  5.4)  which  describes  a  typical 
inspection  activity  in  which  visual  information  is  used  to  trigger  further  investigation  using  other  senses" 


Figure  5.4  Inspection  Model  Flowchart  (Lock  and  Strutt.  1985) 

(Lock  and  Strutt.  1985,  p.  71).  they  note  that  while  particularly  suited  to  area  checks,  the  scenario  is 
generally  applicable  to  a  wide  range  of  inspection  tasks.  These  authors  then  analyze  the  How  chart  for 
error-likely  situations  and  they  identify  six  potential  errors  in  the  inspection  process; 
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Wrong  execution  of  either  of  the  two  tasks:  "identify  next 
inspection"  or  "move  to  location." 

Not  seeing  a  defect  when  one  exists. 

If  human  induced,  due  to  either  "forgetting  to  cover  an 
area"  or  "covering  the  area  inadequately".  May  also  be  a 
schedule  error. 

Error  of  Engineering  Judgement  (E4)  An  error  in  deciding  whether  the  area  in  which  a  defect 

is  found  is  significant  or  not. 

Errors  in  the  Maintenance  Card  System  (E5)  Arises  because  the  work  cards  themselves  may  not  be 

used  to  note  defects  on  the  hangar  floor  immediately  as 
they  are  found. 

Error  in  Noting  Defect  (E6)  The  error  is  noted  incorrectly  or  not  noted  at  all. 

The  authors  recognize  that  these  errors  may  co-occur  to  form  compound  errors.  Lock  and  Strutt  (1985) 
take  a  fault  tree  analysis  approach  to  the  inspection  process  with  "inspection  failure"  as  the  top  event 
(Figure  5,5).  They  note  the  difficulty  of  quantifying  the  probabilities  needed  for  this  type  of  analysis  and 
make  the  necessary  assumptions  (i.e.,  indicating  performance-shaping  factors  relevant  to  inspection, 
estimating  their  relevance  at  each  step,  and  estimating  probabilities  of  detection  at  different  conditions  in 
the  model).  Five  performance-shaping  factors  (PSFs)  were  identified  as  relevant  to  aircraft  inspection: 
accessibility  of  the  aircraft  area,  lighting  (general  area),  access  and  eyeball  enhancement  tools,  motivation 
and  attitude,  and  work  method  (Lock  and  Strutt,  1985).  These  PSFs  were  given  relative  weights  to 
indicate  their  importance  for  each  step  in  the  inspection  process.  The  authors  propose,  but  do  not  actually 
perform,  the  fault  tree  analysis. 

Drury  (1991)  developed  an  error  taxonomy  from  the  failure  modes  of  each  task  in  aircraft  inspection.  This 
taxonomy  has  been  developed  based  on  the  recognition  that  a  pro-active  approach  to  error  control  is 
needed  to  help  identify  potential  errors.  Thus,  the  taxonomy  is  aimed  at  the  phenotypes  of  error 
(Hollnagel,  1989),  that  is,  observed  errors.  Using  the  generic  function  description  of  the  maintenance  and 
inspection  system  (Drury,  et  al.,  1990),  the  goal  or  outcome  of  each  function  was  postulated  as  shown  in 
Table  5.7.  These  outcomes  then  form  the  basis  for  identifying  the  failure  modes  of  the  task.  Towards 
this  end,  the  tasks  within  each  function  were  listed  and  the  failure  modes  for  each  identified.  These 
included  operational  error  data  obtained  from  observations  of  aircraft  in.spectors,  and  discussions  with 
inspectors,  supervisors,  and  quality  control  personnel  involved  in  the  aircraft  maintenance  task,  over  a 
period  of  two  years  (Drury,  Prabhu  and  Gramopadhye,  1990;  Drury,  1991).  A  sample  of  the  error 
taxonomy  (Drury,  1991)  is  shown  in  Table  5.9. 

The  error  framework  developed  by  Rouse  and  Rou.se  (1983)  has  been  u.sed  to  record  and  analyze  human 
errors  in  several  contexts:  (1)  detection,  diagnosis  and  compensation  of  engine  control  room  failures  in 
a  supertanker  (van  Eckhout  and  Rouse.  1981),  (2)  human  errors  in  troublesluxiting  live  aircraft  power 
plants  (Johnson  and  Rou.se,  1982),  (3)  aircraft  pilots  in  mis.sion  flights  (Rouse,  Rou.se.  and  Hammer.  1982). 
Results  of  these  studies  have  been  applied  to  the  improvement  of  training  programs  and  the  development 
of  checklists  and  other  deci.sion  aids.  Drury  (1991)  has  shown  how  this  scheme  may  be  used  to  classify 
errors  occurring  in  both  visual  and  NDT  inspection  tasks  (see  Table  5.10). 


Schedule  Error  (El) 

Inspection  Error  (E2) 
Inspection  Error  (E3) 
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Figure  5.5  Inspection  Error  Fault  Tree  (Lock  and  Strutt,  1985) 

5.3.2.4  An  Approach  to  Aircraft  Inspection  and  Maintenance  Error  Management 

Error  management  may  be  considered  as  a  three  part  objective.  Errors  which  are  evident  in  an  operational 
system  (error  phenotypes)  must  be  identified  and  controlled.  Secondly,  in  order  to  reduce  the  likelihood 
of  unanticipated  error  situations,  errors  must  be  predicted  and  systems  must  be  designed  to  be  error 
tolerant.  Thirdly,  error  reporting  systems  must  provide  error  and  contextual  information  in  a  form  which 
is  appropriate  as  feedback  to  personnel.  Operators  may  then  use  this  information  to  adjust  their  error 
control  and  prevention  strategies  or  alter  environmental  characteristics.  This  section  presents  strategies 
for  error  control  and  prevention  through  error-tolerant  systems.  Finally,  the  need  for  a  context-sensitive 
error  reporting  scheme  is  discussed.  Error  phenotypes  (Hollnagel,  1989),  the  specific,  ob.servable  errors 
in  a  system,  provide  the  foundation  for  error  control.  Error  prevention  and  the  development  of  design 
principles  for  error  avoidance  rely  on  genotype  identification  (Hollnagel,  1989),  associated  behavioral 
mechanisms,  and  their  interaction  with  system  characteristics  (Rasmussen  and  Vicente,  1989).  Here,  error 
phenotypes  are  obtained  empirically  and  from  a  failure-mode-and-effects  analysis  of  task  and 
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communication  models.  These  phenotypes  are  considered  in  light  of  their  ability  to  be  self-correcting  and 
the  type  of  error  which  they  represent.  They  are  further  characterized  by  the  relevant  aspects  of  the 
system  components  with  which  they  interact.  The  resulting  list  of  phenotypes,  their  error  correctability 
and  type,  and  the  pertinent  situational  factors,  allow  designers  to  recognize  these  errors  and  design  control 
mechanisms  to  mitigate  their  effects.  Rasmussen  and  Vicente’s  (1989)  methodology  is  used  to  identify 
genotypes  associated  with  each  phenotype.  This  methodology  yields  the  mechanisms  of  error  formation 


within  the  task  context. 

1  TASK 

ERROR(S) 

1  TASK  1  -  INITIATE  | 

1.1  Correct  instructions  written. 

1.1.1  Incorrect  instnjctions. 

1 . 1 .2  Incomplete  instructions. 

1.1.3  No  instructions  available. 

1 .2  Correct  equipment  procured. 

1 .2.1  Incorrect  equipment. 

1 .2.2  Equipment  not  procured. 

1 .3  Inspector  gets  instnjctions. 

1 .3.1  Fails  to  get  instnjctions. 

1.4  Inspector  reads  instructions. 

1 .4.1  Fails  to  read  instructions. 

1 .4.2  Partially  reads  instnjctions. 

1 .5  Inspector  understands 

instructions. 

1.5.1  Fails  to  understand  instnjctions. 

1 .5.2  Misinterprets  instnjctions. 

1 .5.3  Does  not  act  on  instnjctions. 

1 .6  Correct  equipment  available. 

1 .6.1  Correct  equipment  not  available. 

1.6.2  Equipment  is  incomplete. 

1 .6.3  Equipment  is  not  working. 

1 .7  Inspector  gets  equipment. 

1.7.1  Gets  wrong  equipment. 

1 .7.2  Gets  incomplete  equipment. 

1 .7.3  Gets  non-working  equipment 

1 .8  Inspector  checks/calibrates 

equipment. 

1 .8.1  Fails  to  check/calibrate. 

1 .8.2  Checks/calibrate  incorrectly. 

Table  5.9  Sample  of  Aircraft  Maintenance  and  Inspection  Errors  by  Task  Step 


This  information  in  conjunction  with  consideration  of  influencing  situational  variables  can  predict  the 
forms  of  novel  errors  and  suggest  design  principles  to  prevent  error  formation  and/or  contain  error 
propagation. 

5.3.2.4.1  Error  Control  and  Prevention 


Error  control  is  appropriate  for  the  expedient  eradication  or  mitigation  of  eaor- situation  effects.  However, 
there  is  much  wisdom  in  the  adage  "an  ounce  of  prevention  is  worth  a  pound  of  cure:"  error  nrevention 
is  more  efficacious  than  error  control.  Error  prevention  requires  error  prediction  and  the  design  of  error- 
tolerant  systems. 
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LEVEL  OE  PROCESSING 

POSSIBLE  ERRORS 

1.  Observabon  of  System  State 

Fails  to  read  display  correctly. 

2.  Choice  of  hypothesis 

Instrument  will  not  calibrate,  inspector  assumes  battery  too  low. 

3.  Test  of  hypothesis 

Fails  to  use  knowledge  of  NiCads  to  test. 

4.  Choice  of  goal 

Decides  to  search  for  new  battery. 

5.  Choice  of  procedure 

Calibrates  for  wrong  frequency. 

6.  Execubon  of  procedure 

Omits  range  calibration  step. 

Table  5.10  Hxample  of  Possible  Hrrors  forTxsk  Step  of  Calibrate  NDI  Kquipment  (from  Drury,  1991) 


Error  control  strategies  can  be  derived  by  cla.ssifying  error  phenotypes  according  to  components  of  the 
system  model  (see  Figure  5.3)  and  according  to  Rasmussen  and  Vicente’s  (1989)  sy.stemic  error 
mechanisms.  This  classification  framework  aids  in  suggesting  intervention  strategies  appropriate  to  the 
error  and  the  system  components  involved.  The  system  model  provides  a  useful  means  of  classifying 
observed  errors  and  relating  them  to  specific  human  factors  interventions.  There  are  a  number  of 
personnel  factors  of  general  importance  to  controlling  errors.  Personnel  interactions  are  extremely 
important  aspects  of  the  performance  of  the  inspection  and  maintenance  tasks.  Tliese  interactions  can  be 
immediate  but  are  al.so  accompli.shed  through  the  use  of  forms  and  notes  which  allow  per.sonnel  to 
communicate  with  fewer  temporal  and  spatial  constraints.  Communication  is  information  transferred 
between  not  only  personnel  but  between  personnel  and  documentation.  This  extension  of  the  common 
use  of  "communication”  is  logical  given  that  documentation  can  be  considered  as  a  limited,  static 
representation  of  some  individual’s  (or  group’s)  knowledge.  Equipment  should  he  designed  to  .support 
task  requirements  and  accommodate  human  information  processing  characleri.slics.  The  job  and  the 
individual  tasks  should  be  designed  .such  that  they  can  be  accomplished  at  the  desired  level  of 
performance,  for  the  desired  duration  of  performance,  without  physical  or  affective  stress.  Tlie  physical 
and  organizational  environments  .should  be  de.signed  to  enhance  task  performance  and  ensure  the  safety 
and  motivation  of  personnel. 

Various  intervention  strategies  have  been  suggested  for  the  control  and  prevention  of  errors.  Rouse  (198.^) 
identifies  five  general  interventions  and  propo.ses  a  mathematical  model  for  de.scribing  optimal  re.source 
allocation  among  the  strategies.  These  live  general  categories  are  al.so  reliected  in  the  more  detailed  listing 
of  intervention  strategies  proffered  by  Drury,  et  al.,  ( 1990).  These  interventions  have  been  tailored  to  the 
aircraft  inspection  context  and  were  cla.ssified  as  either  short-term  or  long-term  strategies.  Tlie 
intervention  .strategies  from  the.se  two  .sources  are  de.scribed  in  detail  in  Tables  5.11,  5.12,  and  5.13. 
Table  5.11  pre.sents  a  compilation  of  the  intervention  strategies  and  design  guidelines  propo.sed  by 
Rasmu.ssen  and  Vicente  (1989),  Drury,  et  a!..  (1990),  and  Rouse  (198.‘>).  Tlie.se  intervention  strategies  and 
guidelines  are  classified  by  the  level  of  cognitive  control  (Rasmus.sen,  1986)  which  they  affect  and  the 
type  of  systemic  error  (Ra.smu.s.sen  and  Vicente.  1989)  they  address  (see  Table  5.12).  Intervention 
strategies  can  also  be  cla.s,sified  by  the  component(s)  of  the  aircraft  inspection  and  maintenance  system 
they  alter.  Table  5.13  presents  the  compiled  intervention  strategies  and  design  guidelines  cla.ssified  by 
levels  of  cognitive  control,  sy.stemic  error  and  system  component.  Turther  refinement  of  classification 
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within  system  components  (see  Table  5.8)  is  possible  with  the  aid  of  a  more  detailed  decomposition  of 
these  components  (see  Latorella  and  Drury,  1991). 


SHORT-TERM  INTERVENTIONS  (Shepherd,  et  al.,  1991) 

1 .  Worksheet  design 

2.  NDI  equipment  calibration  procedures 

3.  NOI  equipment  interface 

4.  NOI  equipment  labeling  of  standards 

5.  Support  stands 

6.  Area  localization  aids 

7.  Stands/areas  for  NDI  equipment 

8.  Improved  lighting 

9.  Optical  enhancement 

10.  Improved  NDI  templates 

1 1 .  Standards  available  at  the  workplace 

12.  Pattern  recognition,  job  aids 

13.  Improved  defect  recording 

14.  Hands-free  defect  recording 

15.  Prevention  of  serial  responding  (inadvertent  signoff) 

16.  Integrated  inspection/repair/buy-back  ■  improve  written  communication 

17.  Integrated  inspection/repair/buy-back  -  improve  verbal  communication 

LONG-TERM  INTERVENTIONS  (Shepherd,  etal,  1991  and  Rouse,  1985) 

1 8.  Identification  of  errors  -  error  reporting 

1 9.  Integrated  information  systems  (feedback,  feedfonward,  directive) 

20.  T  raining 

2 1 .  Selection/placement 

ERROR  REDUCTION  RESOURCES  (Rouse,  198S)(also  notes  training  and  selection) 

22.  Equipment  design 

23.  Job  design 

24.  Aiding 

RASMUSSEN'S  'COPING'*  GUIDELINES 
(Rasmussen  and  Vicente,  1989) 

25.  Make  limits  of  acceptable  performance  visible  while  still  reversible. 

26.  Provide  feedback  on  the  effects  of  actions  to  cope  with  time  delay. 

27.  Make  latent  conditional  constraints  on  actions  visible. 

28.  Make  cues  for  action  put  only  convenient  signs,  but  also  represent  the  necessary 
preconditions  for  their  validity  (symbolic). 

29.  Supply  operators  with  tools  to  make  experiments  and  test  hypotheses. 

30.  Allow  monitoring  of  activities  by  overview  displays. 

31.  Cues  for  action  should  be  integrated  patterns  based  on  determining  attributes  (symbolic  representations). 

32.  Support  memory  with  extemalization  of  effective  mental  models. 

33.  Present  information  at  level  most  appropriate  for  decision  making. 

34.  Present  information  embedded  in  a  stnjcture  that  can  serve  as  an  externalized 
mental  model. 

35.  Support  memory  of  items,  acts,  and  data  which  are  not  integrated  into  the  task. 


Table  5.11  Error  Management  Strategies 

The  above  methodology  was  developed  to  control  errors,  i.e.,  for  error  phenotypes  which  are  observable 
errors  in  the  system.  An  extension  of  this  methodology  provides  a  means  by  which  intervention  strategies 
can  be  identified  to  control  unanticipated  errors  once  they  occur.  In  this  extension,  error  genotypes,  rather 
than  the  aforementioned  phenotypes,  are  classified  according  to  the  system  model,  using  Rasmussen  and 
Vicente’s  (1989)  systemic  error  categories  and  Rasmussen’s  levels  of  cognitive  control  (Skill,  Rule, 
Knowledge).  This  characterization  of  error  genotypes  allows  prediction  of  possible,  but  so  far 
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unanticipated,  error  phenotypes.  Unanticipated  errors  can  be  predicted  by  considering  tasks  at  each  level 
of  cognitive  control  and  each  error  mechanisms’  possible  perturbation  of  performance  within  the  context 
of  the  specific  system  components  involved.  Given  an  error  genotype  cell,  intervention  strategies  (which 
also  have  been  classified  by  system  component,  systemic  error  mechanism,  and  cognitive  control  level  (see 
Table  5.13)  can  be  identified  for  its  control. 


SYSTEMIC 

ERRORS 

LEVELS  OF  COGNITIVE  CONTROL 

SKILL 

RULE 

KNOWLEDGE 

Learning  and  Adaptation 

6,  11.  12,  20,  22.  24.  25 

1,2,  11, 20,  22,  24,  28 

1,  13,  16,  17,  18,  19,  20, 

22,  24,  26,  27,  29 

Interference  Among 
Competing  Control 
Structuree 

12,  14,  23,  24,  30 

3,  11,  20,  22,  23,  24,  31 

1,  3,  15,  16,  17,  19,  20,  23, 

24,  32 

Lack  of  Reaourcea 

33 

33,  34 

1,  3,  4,  13,  16,  17,  19,  20, 

21,  22,  23,  24,  33,  34 

Stochastic  Variability 

2,  3,  5,  6,  7,  8,  9,  10,  12,  14, 
16,  17,  20,  21,  22,  23,  24, 

35 

2,  11,  14,  16,  17,  20,  21,  22, 
24,  35 

4,  16,  17,  20,  21,  22,  35 

Table  5.12  Error  Management  Strategies  by  Systemic  Error  and  Level  of  Cognitive  Control 


5.3J.4.2  Error  Tolerant  Design  in  the  Aircraft  Inspection  and  Maintenance  System 

An  error  tolerant  system  has  been  defined  as  a  system  which  ensures  that  recovery  from  errors  is  possible, 
in  the  sense  that  actions  are  reversible  and/or  that  the  system  is  resilient  to  inappropriate  actions  (Rouse, 
1985).  Reason  (1990)  suggests  that  one  way  of  making  systems  more  error  tolerant  is  to  identify  "those 
human  failures  most  likely  to  jeopardize  the  integrity  of  the  plant  and  to  defend  against  them  by 
engineered  safety  devices  or  procedures"  (p.  233).  For  example,  the  "30-minute  rule"  allows  nuclear 
power  plant  operators  30  minutes  of  thinking  time  in  an  emergency  through  the  use  of  automatic  systems 
which  can  return  a  plant  to  a  safe  state  without  human  intervention.  Reason  also  notes  that,  where  these 
safety  devices  are  themselves  subject  to  human  errors,  independent,  redundant  systems  should  be  provided 
(p.  233).  The  design  of  error  tolerant  system  procedures  and  devices  can  be  guided  by  the  error  conhol 
and  prediction  framework  previously  described  by  incorporating  interventions  in  plant  and  operating 
procedure  design. 

5.3.2.4.3  An  Approach  to  Reporting  Aircraft  Inspection  and  Maintenance  Errors 

Currently,  error  reports  are  primarily  used  for  documenting  error  situations  for  administrative  purposes  by 
internal  or  external  regulatory  agencies.  There  are  many  different  regulatory  mechanisms  for  reporting 
errors  to  the  FAA.  In  addition,  the  Air  Transport  Association  (ATA)  has  proposed  modifications  to  those. 
All  of  these  reporting  systems  have  the  following  common  features; 
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Table  5.13  Error  Management  Strategies  by  Systemic  Error,  Levels  of  Cognitive  Control,  and 


System  Component  from  Figure  5J 
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1.  They  are  event  driven.  The  system  only  captures  data  when  a  difficulty  arises  or  a  defect  is 
found. 

2.  Aircraft  type  and  structure  serve  as  the  classification  parameters  for  reporting. 

3.  Expert  judgements  of  error  criticality  are  used  to  further  classify  data  and  determine  its  urgency. 

4.  To  some  extent  in  all  systems,  the  feedback  of  digested  data  to  users  is  not  well-engineered. 
Thus,  for  the  end-user  level,  the  data  collection  effort  is  largely  for  naught. 

5.  They  can  result  in  changes  in  maintenance  and  inspection  procedures;  for  example,  by  issuing 
Airworthiness  Directives  (ADs). 

Error  reports  in  maintenance  and  inspection  produced  for  administrative  purposes  are  typically  concerned 
with  establishing  accountability  for  an  error  and  its  consequences  rather  than  understanding  the  causal 
factors  and  situational  context  of  the  error.  This  type  of  information  is  not  appropriate  for  use  as 
performance  feedback  to  inspectors  or  maintenance  personnel,  nor  is  it  helpful  information  for  error 
tolerant  system  design.  Error  reporting  schemes  are  developed  from  within  an  organization  and  therefore 
vary  greatly  among  organizations.  The  framework  of  these  error  reporting  schemes  is  event  driven  and 
developed  iteratively,  thus  additions  are  made  only  with  the  occurrence  of  a  new  error  situation.  To  a 
large  extent,  the  information  recorded  about  a  situation  is  constrained  by  the  format  of  the  error  reporting 
scheme.  For  example,  in  one  error  reporting  scheme,  the  reviewer  is  required  to  attribute  the  error  to 
some  form  of  human  error  unless  the  situation  can  be  described  as  an  "act  of  God"  (Drury,  1991). 
Analysis  of  the  data  collected  by  such  a  scheme  will  invariably  find  the  human  at  fault,  rather  than 
working  conditions,  equipment,  procedures,  or  other  external  factors.  This  biased  representation  has 
serious  implications  for  error  prevention,  especially  considering  that  equipment  design  and  Job  aiding  have 
been  found  to  be  more  efficacious  than  selection  or  training  approaches  in  error  prevention  (Rouse,  1985). 
To  alleviate  the  difficulties  of  inconsistency,  and  provide  an  appropriate  and  useful  structure  for  error  data 
collection,  an  error  reporting  scheme  should  be  developed  from  a  general  theory  of  the  task  and  the  factors 
which  shape  how  the  task  is  performed.  Principally,  the  behavioral  characteristics  of  the  operator,  but 
ideally  also  organizational  environment.  Job  definition,  workspace  design,  and  the  operators’  physical, 
intellectual  and  affective  characteristics  should  be  considered.  Effective  error  categorization  systems  are 
not  only  descriptive  but  are  prescriptive,  providing  information  for  specific  intervention  strategies  (i.e., 
Langan-Fox  and  Empson,  1985  and  Kinney,  ct  al.,  1977). 

As  Rasmussen,  Duncan,  and  Leplat  (1987)  note,  it  is  nece.ssary  to  shift  the  focus  of  analysis  from  the  task 
to  the  interaction  of  the  task  and  the  operator  for  classifying  errors.  Furthermore,  taxonomies  of  human 
error  must  encompass  the  analysis  of  not  only  the  task  characteristics  but  also  the  information  processing 
mechanisms  associated  with  the  subtasks.  It  is  apparent  that  other  situational  characteristics  (i.e., 
environmental  conditions)  are  also  useful  for  the  sensitive  classification  of  errors  (Stager  and  Hameluck, 
1990).  Correlations  of  errors  with  situational  factors,  with  remedies  attempted,  and  with  the  effects  of 
these  remedies,  may  provide  important  feedback  for  identifying  error  .situations,  assessing  error  criticality, 
and  determining  error  consequence-minimizing  solutions.  Doth  error  control  and  error  prevention  would 
benefit  from  an  error  reporting  system  which  captures  the  causal  factors  and  situational  context  of  an  error 
situation. 

Both  the  taxonomic  approach  of  Drury  and  Prabhu  (1991)  and  the  taxonomy  for  error  management 
strategies  developed  here  can  be  used  as  a  basis  for  formulating  error  reporting  .schemes.  Upon 
occurrence,  errors  can  be  classified  by  level  of  cognitive  conhol.  type  of  systemic  error,  and  by  causal  or 
catalytic  elements  of  the  system.  As  previously  mentioned,  the  categories  of  system  elements  can  be 
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refined  as  illustrated  in  Table  5.8  to  provide  a  more  descriptive  error  characterization.  Identification  of 
these  parameters  will  likely  involve  detailed  investigation  of  the  error  situation,  including  extensive 
operator  interviewing.  This  data  store  can  be  analyzed  for  trends  in  error  sequences,  effects  of  different 
intervention  strategies  on  error-type  frequency,  and  for  the  efficacy  of  intervention  strategies  over  all  types 
of  errors.  Identification  of  error  sequences  and  the  effects  and  interactions  of  system  elements  provides 
important  feedback  information  for  performance  and  feedforward  information  for  training,  equipment,  and 
job  design.  A  prototype  error  reporting  system  based  on  the  above  considerations  has  been  proposed  as 
a  short-term  project  with  an  airline  partner. 

5.3J  A  FRAMEWORK  FOR  INFORMATION  ENVIRONMENT  DESIGN  FOR  AIRCRAFT 
INSPECTION 

Inspection  is  information  processing.  Other  aspects  of  the  inspector’s  task,  such  as  physical  access  to  the 
work  and  body  posture  during  work,  are  subordinate  to  this  central  task.  If  information  processing  is  the 
essence  of  inspection,  we  must  examine  the  sources  of  information  used  (and  not  used)  by  the  inspector: 
how  information  is  received,  processed  and  generated.  Hence,  the  inspector’s  information  environment 
is  a  critical  part  of  the  inspection  system. 

Any  system  involving  a  human  is  typically  closed  loop  (e.g.,  Sheridan  and  Ferrell,  1974).  Obvious 
examples  are  in  flying  an  aircraft  or  driving  a  car,  but  the  concept  applies  equally  to  inspection  tasks.  As 
shown  in  Figure  5.6,  the  human  in  the  task  receives  some  instruction,  or  command  input  to  use  systems 
terminology.  The  operator  and  any  associated  machinery  transform  this  command  input  into  a  system 
output.  To  ensure  stable  performance,  the  system  output  is  fed  back  to  the  input  side  of  the  system,  where 
it  is  compared  against  the  command  input.  If  there  is  any  difference  (command  minus  output)  the  system 
responds  so  as  to  reduce  this  difference  to  zero. 

From  the  model  in  Figure  5.6,  it  is 
obvious  that  two  types  of  information 
can  be  distinguished.  The  input  is 
command  information,  while  the  output 
is  feedback  information.  Both  have 
been  shown  to  be  amenable  to 
manipulation  to  improve  system 
performance.  Not  obvious  from 
Figure  5.6  is  that  the  command  input 
may  be  complex,  and  includes  both 
what  needs  to  be  accomplished  and 
help  in  the  accomplishment.  Thus, 
input  may  give  both  directive  and 
feedforward  information.  A  work  card 
may  contain  "detailed  inspection  of 
upper  lap  joint"  in  a  specified  area 
(directive)  and  "check  particularly  for 
corrosion  between  stations  28(X)  and 
2840"  (feedforward).  Thus,  there  are 
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Figure  5.6  Closed-Loop  Control 


95 


Chapter  Five 


really  three  potential  parts  to  the  information  environment:  directive  information,  feedforward  information, 
and  feedback  information. 

Three  of  the  strongest  influences  found  in  case  studies  of  inspection  performance  are  time  pressure  on  the 
inspector,  feedforward  of  information  to  the  inspector,  and  feedback  of  detailed  performance  measures. 
We  restrict  ourselves  to  examining  the  various  aspects  of  feedforward  and  feedback  information  in  the 
context  of  aircraft  inspection;  the  time  pressure  aspect  is  dealt  with  under  speed  accuracy  tradeoff  in 
Section  53.4. 

In  the  subsequent  sections  we  present  a  model  of  the  information  flow  in  aircraft  inspection.  This  model 
serves  as  the  basis  for  understanding  the  information  environment  that  the  inspector  is  a  part  of.  We  then 
present  two  approaches  to  analyze  the  information  requirements  of  the  inspection  task:  (a)  skill-rule- 
knowledge  (S-R-K)  based  approach,  and  (b)  error  taxonomic  approach.  Finally,  a  study  to  investigate  the 
effect  of  feedback  information  is  described. 

5.33.1  A  Model  of  Information  Flow  in  Aircraft  Inspection 

To  perform  optimally  in  the  system,  the  inspector  has  to  have  access  to  the  relevant  information  and  the 
information  environment  has  to  provide  this  information.  We  have  to  reconcile  the,  perhaps  conflicting, 
issues  of: 

•  What  information  to  present. 

•  When  to  present  this  information. 

•  How  to  present  this  information. 

In  designing  the  flow  of  information,  the  designer  must  take  into  account  human  processing  of  information 
and  the  cognitive  abilities  of  humans.  It  is  important  to  develop  a  model  of  the  information  environment 
in  order  to  analyze  the  current  system  and  propose  design  changes  based  on  identified  problems.  Towards 
this  end  we  propose  a  feedforward/feedback  information  model  of  aircraft  inspection  (see  Figure  5.7). 
This  model  represents  both  the  physical  work  flow  and  the  information  flow.  It  also  highlights  the 
cognitive  aspects  of  the  inspection  task  and  its  interaction  with  the  information  environment. 

This  model  allows  us  to  target  the  components  of  feedforward  (training,  documents,  etc.)  and  feedback 
(missed  defects,  defect  rate,  etc.)  that  have  to  be  analyzed  for  efficient  design  of  the  information 
environment. 

5.33.1.1  Feedforward  Information 

From  the  model  (Figure  5.7),  feedforward  information  to  the  iaspcctor  is  seen  to  come  from  the  following 
sources: 

1.  Initial  Training 

2.  Manufacturer/FAA/ Airline  Operator  documents. 

3.  On-the-Job  experience  on  a  particular  aircraft. 

4.  Information  gathered  from  co-workers. 
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Figure  5.7  Model  of  Information  Flow  in  Aircraft  Maintenance  and  Inspection  (Drury  and  ftabhu, 
1991) 


5.  Command  information  in  the  form  of  standards. 

6.  Utilization  of  understanding  about  the  fault  causation  mechanism  in  an  aircraft. 

Initial  Training.  Taylor  (1990)  notes  aircraft  orientation  training  for  new  mechanics,  at  large  sites. 
However,  smaller  sites  had  no  formal  training  programs  in  place.  No  formal  inspection  training  programs 
were  observed  or  reported  at  any  of  the  airlines.  Typically,  inspectors  hold  an  A  and  P  license  and  have 
maintenance  experience.  Taylor  (1990)  found  that  the  current  hangar  maintenance  organization  has  a 
bi-modal  experience  distribution  of  .TO  plus  years  and  three  or  fewer  years.  The  inspection  group  is 
expected  to  have  a  similar  distribution  with  three  to  five  years  added  to  the  lower  value. 


The  current  state  of  training  places  much  emphasis  on  both  the  procedural  aspects  of  the  task  (e.g.,  how 
to  set  up  for  an  X-ray  inspection  of  an  aileron),  and  on  the  diagnosis  of  the  causes  of  problems  from 
symptoms  (e.g.,  troubleshooting  an  elevator  control  circuit).  However,  the  inspectors  we  have  studied  in 
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our  task  analysis  work  have  been  less  well  trained  in  the  cognitive  aspects  of  visual  inspection  itself.  How 
do  you  search  an  array  of  rivets  —  by  columns,  by  rows,  or  by  blocks?  How  do  you  judge  whether 
corrosion  is  severe  enough  to  be  reported? 

Most  of  the  training  is  on  the  Job  where  an  experienced  inspector  puts  the  novice  through  his  paces  and 
shows  him  the  various  aspects  of  inspection.  This  is  highly  realistic  but  uncontrolled  and  there  is  a  high 
likelihood  for  development  of  inconsistent  inspection  practices.  Our  experience  in  training  inspectors  in 
manufacturing  industries  (Kleiner,  1983)  has  shown  that  a  more  controlled  training  environment  produces 
better  inspectors.  If  training  is  entirely  on-the-job,  then  two  of  the  main  determinants  of  the  training 
program-what  the  trainee  sees  and  what  feedback  is  given-are  a  matter  of  chance;  i.e.,  of  which  particular 
defects  are  present  in  the  particular  aircraft  inspected. 

We  need  to  develop  training  procedures  for  the  search  and  decision  making  components  of  aircraft 
inspection  by  using  human  factors  techniques  that  include  cueing,  feedback,  active  training,  and 
progressive  part-training  as  suggested  by  Drury  and  Gramopadhye  (1990)  and  detailed  in  Section  5.3.5. 
It  has  been  found  that  off-line  controlled  training  successfully  transfers  to  the  more  complex  on-the-job 
environment.  The  trainee  is  prepared  to  make  maximum  use  of  what  is  seen  on  the  job,  rather  than 
confining  the  learning  process  to  trial  and  enor.  Because  of  the  controlled  and  concentrated  training 
experience,  trainees  can  progress  faster  to  the  same  level  as  experienced  inspectors. 

Documentation.  There  is  an  immense  amount  of  potentially  useful  information  available  both  in  paper 
(hard  copies)  and  paperless  (computer,  microfiches)  form.  We  list  below  some  of  the  important 
documents  that  form  the  information  environment.  Note  that  this  is  not  a  complete  listing  of  all  available 
documents. 

The  documents  are  generated  by  a  triad  consisting  of  the  Federal  Aviation  Administration  (FAA),  aircraft 
manufacturers,  and  aircraft  operators.  There  is  a  complex,  multi-dimensional  interaction  in  the  flow  of 
data  between  these  three.  Manufacturers  require  feedback  from  operators  to  determine  acceptability  and 
reliability  of  a  product  and  its  components.  Airlines  require  product  support  information  from  the 
manufacturer.  The  FAA  requires  data  from  both  the  airlines  and  the  manufacturers  concerning  product 
reliability  and  safety  issues.  The  Air  Transport  Association  (AT A)  coordinates  the  flow  of  data  among  the 
three  triad  members  (Shepherd,  1990). 

We  have  to  understand  the  problems  created  by  the  mismatches  between  the  needs  of  the  inspector  (who 
is  looking  for  information)  and  the  design  of  the  documents  (that  present  data).  There  is  a  critical  need 
for  usable  knowledge,  which  gets  translated  to  utilized  information  on  the  job.  From  a  document  design 
viewpoint  we  have  to  focus  on  creating  usable  documents.  Information  flow  design  and  system  design 
should  ensure  the  availability  of  documents  at  the  right  place  at  the  right  time.  The  demonstration  project 
presented  in  Section  5.2.1  is  an  example  of  applying  document  design  techniques  to  one  type  of 
document,  the  workcard. 

Experience  on  a  Specific  Aircraft  Type.  Aircraft  at  a  maintenance  facility  are  serviced  over  various  lengths 
of  time  depending  on  the  type  of  service.  The  transfer  of  an  aircraft  to  a  different  facility  (other  than  the 
one  it  normally  goes  to)  is  very  rare  and  occurs  in  case  of  contingencies  or  in  case  of  heavy  workload  at 
the  regular  facility.  Similarly,  movement  of  personnel  between  different  facilities  is  very  low.  Thus,  most 
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maintenance  and  inspection  personnel  accumulate  experience  on  a  particular  type  of  aircraft.  The  effect 
of  such  job  specialization  on  the  occasional  inspection  of  a  different  aircraft  type  has  not  been  studied. 

Knowledge  about  the  aircraft  is  accumulated  over  a  period  of  time  through  on-the-job  work.  Experienced 
inspectors  gradually  develop  an  understanding  of  the  cause-effect  relationship  of  defects  and  also  know 
what  to  look  for  and  where.  Thus,  there  is  a  store  of  distributed  knowledge  or  expertise  residing  in  the 
inspection  organization.  Individual  inspectors  normally  have  access  to  this  distributed  knowledge  through 
informal  contacts  with  fellow  inspectors,  which  leads  us  to  the  next  section. 

Information  from  Co-Workers.  The  relevant  relationships  in  heavy  maintenance  have  been  identified  by 
Taylor  (1990),  to  include: 

1 .  Superiors  with  subordinates 

2.  Members  of  same  group  with  one  another 

3.  Members  of  different  work  groups 

4.  People  inside  enterprise  interacting  with  people  outside  that  system. 

Airline  inspectors  typically  work  independently  and  occasionally  in  teams  of  two.  The  frequency  of 
formal  meetings  amongst  inspectors  varies  from  airline  to  airline.  In  one  airline,  weekly  safety  meetings 
are  held  where  any  communications  from  management  are  conveyed  to  the  inspectors.  In  another  case 
there  is  a  daily  meeting  at  the  beginning  of  the  shift  where  the  day’s  work  and  assignments  are  discussed. 
Drury,  et  al.,  (1990),  during  the  task  analysis  of  inspection  in  the  airl’ne  industry,  found  few  formal 
meetings  of  mechanics  or  inspectors  despite  frequent  informal  contact  among  inspectors,  and  less  frequent 
contact  between  inspectors  and  mechanics. 

Contact  between  inspectors,  in  different  shifts,  was  observed  at  some  sites  where  shifts  overlapped  by  an 
hour  or  so.  The  mechanics  and  inspectors  contact  each  other  for  buy-back  or  for  approval  of  a  repair. 
This  contact  for  advice/instruction  is  the  only  formal  information  exchange  between  the  inspector  and  the 
mechanic.  There  appears  to  be  no  formally  organized  forum  that  can  channel  the  distributed  knowledge 
for  more  efficient  access  by  individuals  who  need  this  information. 

Mechanics  who  find  faults  during  scheduled  maintenance  notify  the  inspectors.  Thus,  an  informal  system 
of  communication  exists.  However,  there  are  various  ways  in  which  such  a  system  can  break  down.  An 
experienced  inspector  might  know,  for  example,  that  the  tine  maintenance  people  have  in  the  past 
improperly  used  magnetic  screws  around  the  landing  light  as  a  contingency  measure.  Thus,  he/she  would 
examine  the  screws  around  the  landing  light  in  view  of  this  knowledge.  A  new  inspector  may  not  have 
had  access  to  this  issue  (which  is  not  mentioned  in  a  workcard  or  any  documentation  elsewhere)  and  could 
fail  to  catch  such  a  fault.  Similarly,  an  inspector  who  documents  a  fault  and  the  inspector  who  approves 
the  repair  done  on  this  fault  may  not  be  the  same  and  thus,  any  inspection  error  in  this  case  goes 
unnoticed  by  the  inspector  because  of  a  lack  of  a  formal  feedback  system. 

Command  Information  with  Comparative  Standards.  There  seem  to  be  almost  no  standards  that  are 
accessible  to  inspectors  for  defects  like  corrosion,  cracks,  dishcd/pooched  rivets,  wear,  component  play, 
etc.  A  small  subset  of  standards  does  exist  with  the  manufacturer,  FAA,  etc.,  but  these  have  not  been 
organized  into  a  scheme  for  utilizing  comparative  standards  on  the  job.  The  closest  inspectors  come  to 
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a  standard  in  visual  inspection  is  to  use  adjacent  areas  to  make  a  comparison,  which  is  not  a  reliable 
method  (Drury,  1991). 

During  a  decision  making  process,  both  the  internal  and  external  retrieval  of  information  is  necessary. 
The  degree  with  which  external  and  internal  retrieval  of  information  is  required  could  be  a  major 
determinant  of  the  strategies  adopted  during  decision  making. 

As  an  example,  during  visual  search  for  corrosion  around  rivets  or  in  a  door  frame  the  inspector  comes 
across  an  indication.  The  inspector  has  to  make  a  judgement  call  whether  this  indication  should  be 
marked  as  a  defect  or  let  go.  If  the  corrosion  is  evident  without  a  doubt,  then  the  decision  process  is 
simple  and  the  task  is  almost  like  a  pure  search  task.  On  the  other  hand,  when  the  evidence  for  corrosion 
in  the  indication  is  not  conclusive,  the  inspector  has  to: 

1.  Retrieve  internal  information  about  instances  of  corrosion  to  make  a  match  (recall  patterns). 

2.  Approach  peers  or  supervisors  for  help  on  judgement. 

3.  Refer  to  comparison  standards  available  at  the  work  point. 

It  has  been  found  that  the  higher  the  information  load  and  the  more  likely  the  chance  of  error,  the  more 
an  operator  is  forced  to  remember  or  recall  information  of  relevance.  Also,  external  information  retrieval 
(from  other  inspectors)  is  a  function  of  the  opjerator’s  perception  of  criticality  of  this  particular  decision 
and  availability  of  inspectors  within  a  reasonable  vicinity.  For  example,  the  inspector  perched  up  on  the 
horizontal  stabilizer  of  a  DC-9  is  less  likely  to  go  down  and  call  a  supervisor  to  come  up  and  have  a  kx)k 
at  an  indication,  particularly  if  he  perceives  that  a  wrong  decision  on  his  part  may  not  be  critical. 

It  has  been  known  for  many  years  that  if  comparison  standards  are  available  at  the  work  point,  more 
accurate  inspection  will  result.  Yet  in  many  cases  such  standards  are  not  available  to  the  aircraft  inspector. 
For  example,  if  the  maximum  allowable  depth  of  a  wear  mark  is  given  as  0.010  inches,  there  is  neither 
a  convenient  way  to  measure  this,  nor  a  readily  available  standard  for  comparison.  Other  examples  are 
play  in  bearings  and  cable  runs,  areas  of  corrosion,  and  looseness  of  rivets.  All  are  considered  to  be 
"judgement  calls"  by  the  inspector,  but  simple  job  aids,  perhaps  as  part  of  the  worksheet,  or  standard 
inspection  tools,  would  remove  a  source  of  uncertainty.  Leaving  standards  to  unaided  human  memory 
may  be  expeditious,  but  it  is  also  unreliable. 

Utilization  of  Understanding  about  the  Fault  Causation  Mechanism  in  Aircraft.  Inspxjction  of  aircraft  is 
largely  composed  of  pure  search  activities  followed  by  decision-making  tasks  whose  output  is  of  the  form 
of  "acceptable/non-acceptable".  However,  some  areas  of  inspection  involve  utilization  of  cues,  knowledge 
of  how  faults  are  caused,  and  knowledge  of  how  the  behavior  of  one  particular  aircraft  component 
indicates  behavior  of  related  components.  Examples  are: 

•  dirt  streaks  around  a  rivet  on  the  fuselage  indicate  a  kwise  rivet, 

•  bulging  of  the  paint  on  the  aircraft  skin  indicates  underlying  corrt)sion, 

•  scraped  paint  at  the  fairings  indicates  underlying  fairings  are  rubbing, 

•  play  at  the  flap  vanes  points  to  worn  out  bearings  or  tracks, 

•  flat  spots  on  the  wheel  indicate  a  possible  problem  with  the  anti-skid  system, 

•  powdery  material  on  the  skin  indicates  {X'obable  corrosion. 
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Use  of  such  indirect  evidence  is  a  powerful  technique  to  enhance  detection  and  discovery  of  a  fault, 
particularly  where  faults  are  not  directly  accessible  to  pure  visual/auditory/tactile  search. 

There  is  a  necessity  to  gather  the  knowledge  required  for  this  indirect  fault  indication  from  experienced 
inspectors  who  understand  the  utilization  of  such  cues.  There  is  also  a  need  to  identify  the  mappings 
between  defects  and  fault  causation  mechanisms  for  a  wide  variety  of  such  defects.  The  decision-making 
activity  can  then  be  converted  to  a  rule-based,  procedural  type  of  task.  Rules  thus  formed  can  be  used 
in  an  effective  training  scheme  to  help  inspectors  increase  the  efficiency  of  the  search  and  decision  making 
process. 

This  approach  can  be  extended  further  to  form  an  inspection  data  base  which  can  be  continually  revised 
and  updated  to  reflect  the  distributed  knowledge  that  exists  not  only  in  a  specific  airline  but  across  all 
airlines.  Such  a  global  knowledge-base  would  thus  receive  its  input  from  experienced  inspectors  all  over 
the  aviation  industry,  thus  consistently  benefitting  all  users.  It  is  also  conceivable  that  an  expert  system 
could  be  developed  that  makes  use  of  such  a  data  base  and  supports  decision-making  tasks.  Such  a  system 
would  support  queries  like: 

•  "I  am  in  the  tail  compartment.  Current  inspection  area  is  aft  of  APU  compartment  bulkhead,  list 
keypoints." 

•  "Inspection  area  is  APU  shroud,  list  past  history  of  cracks.” 

•  "Indications  at  rivet  on  lap  joint  at  stringer  S-34  between  body  station  890  and  900  points  to 
corrosion,  show  graphics  of  likely  corrosion  in  this  area." 

•  "There  is  excessive  play  at  the  flap  vanes,  what  are  the  problems  indicated  by  this.",  etc. 

5.3J.1.2  Feedback  Information 

Feedback  information  in  aircraft  inspection  can  be  used  either  on  the  job  or  in  training.  Use  of  feedback 
on  the  job  has  been  found  to  reduce  the  number  of  false  alarms  as  well  as  reduce  missed  defects.  Training 
schemes  implementing  feedback  have  been  used  to  improve  learning  rates,  to  develop  schemes,  and  for 
the  efficient  transfer  of  training  skills  to  on-the-job  performance. 

On-The-Job  Feedback.  There  seems  to  be  no  systematic  and  obvious  system  in  place  that  provides 
feedback  to  the  inspector.  For  example,  feedback  during  access  can  be  given  by  a  well  designed  workcard 
system  incorporating  unique  landmarks  in  the  figures  (Drury,  1990b).  Feedback  in  search/decision  making 
comes  when  the  inspector  talks  to  a  supervisor  or  a  fellow  inspector  to  confirm  a  borderline  case,  although 
this  occurs  rarely.  Also  rare  is  the  feedback  that  could  come  from  the  repairer  or  the  buy-back  inspector 
who  both  have  potential  data  on  the  fault. 

Feedback  also  seems  to  depend  on  the  type  of  defect.  Airlines  have  a  system  to  classify  the  various 
defects  found  during  inspection/maintenance.  There  are  specific  rules  by  the  FAA  for  this  classification. 
Normally,  defects  get  classified  in  three  broad  categories:  A,  B,  and  C.  Type  "A”  defects  are  the  most 
critical  ones  and  have  to  be  immediately  corrected.  Type  "B"  defects  are  corrected  immediately,  or  the 
maintenance  action  is  deferred  to  a  pre-specified  time  based  on  current  and  projected  workload.  The  "C" 
defects  are  generally  deferred  to  the  next  inspection.  Thus,  there  exists  a  possibility  of  feedback  in  the 
case  of  "A"  defects,  and  some  "B"  defects,  because  of  the  time  frame  within  which  maintenance  action 
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is  taken.  This  would  normally  occur  through  buy-back  inspection.  However,  even  this  opportunity  would 
be  lost  if  the  buy-back  inspector  is  different  from  the  one  who  wrote  the  non-routine  defect  item. 

There  is  very  little  feedback  on  any  defect  that  the  inspector  misses.  This  feedback  can  only  occur 
through  audits  and  quality  control  inspections,  but  these  systems  do  not  ensure  consistent  feedback  to  all 
inspectors  on  a  regular  basis. 

At  this  point  we  have  to  also  recognize  that,  although  it  is  very  desirable  to  provide  feedback,  there  are 
bound  to  be  instances  where  this  would  be  economically  infeasible,  and  in  some  cases  impossible,  due  to 
the  nature  of  the  task.  For  example,  providing  regular  feedback  on  missed  defects  is  not  viable,  as  it 
would  involve  re-inspection  similar  to  auditing  on  a  regular  basis.  Similarly,  having  a  system  that  calls 
for  feedback  on  every  defect  may  be  too  expensive  due  to  time  factors  and  logistics.  In  such  cases, 
alternate  schemes  like  pxiriodic  re-training  or  off-line  feedback  could  be  utilized  to  re-calibrate  inspectors. 

Feedback  in  Traitung.  As  explained  in  the  earlier  section,  the  feedback  in  aircraft  inspection  is  relatively 
scarce,  and  on  the  occasions  that  the  inspector  gets  feedback  (e.g.,  an  audit),  it  is  delayed  in  time. 
Delayed  feedback  makes  learning  by  practice  alone  difficult  (Woods,  1989). 

The  use  of  knowledge  of  results  (feedback)  in  training  is  well  documented.  The  trainee  needs  rapid, 
accurate  feedback  in  order  to  correctly  classify  a  defect  or  to  know  whether  a  search  pattern  was  effective. 
However,  when  training  is  completed,  feedback  is  rare.  The  training  program  should  start  with  rapid, 
frequent  feedback,  and  gradually  delay  this  until  the  "working"  level  is  reached.  More  feedback  beyond 
the  end  of  the  training  program  will  help  to  keep  the  inspector  calibrated  (Drury  and  Kleiner,  1990). 

We  see  that  there  is  a  great  deal  of  research  support  to  indicate  that  use  of  feedback  in  initial  haining  is 
beneficial.  From  the  airline  inspection  context  this  points  to  the  necessity  of  developing  a  haining 
methodology  that  incorporates  performance  feedback.  Drury  and  Gramopadhye  ( 1990)  have  demonstrated 
a  training  scheme  for  gamma  ray  inspection  of  a  nozzle  guide  vane  area  of  a  JT9D  engine.  This  includes 
part  naming  and  defect  naming  (cueing  and  active  response),  search,  and  decision  haining.  Feedback  is 
used  judiciously  in  this  haining  scheme  to  help  the  hainee  to  build  a  schema. 

5.3J.2  Analysis  of  Information  Requirements:  An  S-R-K  Approach 

So  far  it  has  been  established  that  (a)  errors  in  aircraft  inspection  are  costly,  and  therefore  must  be 
minimized,  (b)  human  performance  limitations  can,  and  do,  result  in  inspection  errors,  and  (c)  provision 
of  information  in  the  correct  form  (physical  and  cognitive  aspects)  is  critical  to  reducing  human  errors. 

For  effective  use  of  feedforward  and  feedback  information,  the  information  requirements  of  human 
inspection  have  to  be  identified.  Furthermore,  the  information  needs  of  experts  and  novices  may  be  very 
different.  Thus,  we  can  posit  that  studying  the  behavior  of  the  human  inspector  interacting  with  the 
system  (while  performing  the  inspection)  will  help  identify  possible  information  support  points,  as  well 
as  provide  guidance  to  the  type  of  information  (either  feedforward  or  feedback)  that  is  needed  at  these 
points.  The  skill-rule-knowledge  based  hierarchy  of  Rasmu.ssen  (1983)  presented  in  Section  5J.2.1.2 
affords  us  a  robust  framework  within  which  this  analysis  can  be  carried  out,  and  will  be  mapped  onto  both 
visual  inspection  and  NDI. 
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5.3J.2.1  Visual  Inspection 

Search  and  decision  making  form  the  critical  components  of  visual  inspection.  The  search  component  can 
be  further  decomposed  into  pre-attentive  search,  and  a  detailed  search  consisting  of  foveal  (pure  search 
or  search  plus  decision  making)  and  extra-foveal  processes.  Similarly,  NDI  can  be  decomposed  into  three 
broad  stages:  calibration,  probe  movement,  and  display  interpretation.  Identification  of  the  behaviors 
associated  with  each  of  these  subtasks  results  in  a  many  to  many  mapping  as  seen  in  Table  5.14  (Visual 
Search)  and  Table  5.15  (NDI).  These  mappings  have  been  identified  for  an  expert  inspector.  An 
interesting  aspect  of  these  mappings  is  the  existence  of  relatively  few  knowledge-based  behaviors  exhibited 
by  the  exp)ert  inspector.  This  seems  logical  since  there  is  less  problem-solving  or  active  reasoning  in 
aircraft  inspection  and  more  detection,  identification,  and  classification. 


VISUAL 

INSPECTION 

PROCESSES 

BEHAVIOR  CATEGORIES 

SKILL-BASED 

RULE-BASED 

KNOWLEDGE-BASED 

PRE-ATTENTIVE  SEARCH 

Scan  and 

Detect 

FOVEAL  (PURE  SEARCH) 

Fixate  and 

Detect 

FOVEAL  DECISION 

Identify  and 

Classify 

EXTRA-FOVEAL  SEARCH 

Trigger  move 
to  next  area 

DECISION-MAKING  (OUTSIDE 
OF  SEARCH) 

Move  to  next  area, 

Rules  of  what  to  look 
for 

Reason  and  Decide 

Table  5.14  Mapping  a  Visual  Inspection  Task  to  Cognitive  Behavior  for  an  Expert  Inspector 


NDI  PROCESSES 

BEHAVIOR  CATEGORIES 

SKILL-BASED 

RULE-BASED 

KNOWLEDGE-BASED 

CALIBRATION 

Probe  Movement  Over 
Test  Specimen 

Calibration  Procedures 

PROBE  MOVEMENT 

Tracking  Along 

Desired  Path 

Supportive  Mode 
Identifying  Boundary 
Conditions 

DISPLAY  INTERPRETATION 

Interpreting  Familiar 
Signal 

Interpreting  Unfamiliar, 

Unanticipated  Signals 

Table  5.15  Mapping  an  NDI  Procc.ss  to  Cognitive  Behavior  for  an  Expert  In.spector 


The  SRK  framework  also  aids  understanding  of  how  behavior  will  be  qualitatively  modified  as  the 
inspector  goes  from  a  novice  to  an  expert.  Thus,  although  both  the  novice  and  the  expert  exhibit,  say. 
rule-based  behavior,  the  behavior  of  the  expert  will  be  qualitatively  different  from  the  novice  (Sanderson 
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and  Harwood,  1988).  In  Table  5.16  we  have  mapped  a  specific  visual  inspection  task  (inspection  of 
rivets)  to  the  SRK  framework,  to  represent  the  performance  of  an  expert  inspector.  We  can  expect  that 
some  of  the  defects  identified  at  the  skill-based  and  rule-based  levels  by  the  expert  will  be  identified  at 
the  rule-based  and  knowledge-based  levels  by  the  novice,  indicating  a  rightward  shift  on  Table  5.16, 
corresponding  to  an  upward  movement  on  the  SRK  hierarchy.  Thus,  this  analysis  points  to  the  need  for 
different  levels  of  information  support  for  the  expert  and  the  novice  inspector.  It  can  also  provide 
guidelines  to  define  training  requirements  for  novice  inspectors  ba.sed  on  identifying  expert  inspector 
behaviors. 


VISUAL 

INSPECTION 

PROCESSES 

BEHAVIOR  CATEGORIES 

SKILL-BASED 

RULE-BASED 

KNOWLEDGE- 

BASED 

PRE-ATTENTIVE 

SEARCH 

•  Missing  rivet 

•  Hole  in  skin 

FOVEAL (PURE 
SEARCH) 

•  Missing  rivet 

•  Hole  in  skin 

•  Deep  dents 

•  Large  cracks 

•  Prominent 
corrosion 

FOVEAL  DECISION 

•  Borderline  corrosion 

•  Slight  wear 

•  Dished  rivets 

•  Ripples  in  skin 

•  Small  cracks 

EXTRA-FOVEAL 

SEARCH 

•  Chipped  paint  in 
periphery  leads 
to  next  fixation 

DECISION-MAKING 
(OUTSIDE  OF 

SEARCH) 

•  Streaks  around 
rivets  trigger 
inspection  for 
loose  rivets 

•  Powdery 
contamination 
triggers  search 
for  corrosion 

•  Borderline 
defects 

•  Defect  type  not 
listed 

•  Use  of  meta¬ 
knowledge 

Table  5.16  Visual  Inspection  of  Rivets;  Cognitive  Behaviors  for  Different  Defect  Types 

Tables  5.14,  5.15,  and  5.16  also  indicate  the  large  role  that  skill-based  and  rule-ba.sed  behaviors  play  in 
visual  inspection.  The  visual  search  part  of  visual  inspection  is  seen  to  be  entirely  skill  and  rule-based 
for  the  expert  inspector  (or  after  training  to  criteria).  The  skill-based  behavior  can  be  associated  to  the 
scanning,  fixating,  and  detecting  activities  (see  Table  5.14).  Since  skill-based  performance  is  essentially 
unconscious  and  feedforward  controlled,  we  can  conclude  that  the  information  aid  for  this  part  of  the 
visual  search  should  be  something  that  does  not  require  active  conscious  u.se  by  the  inspector.  This  points 
to  visual  environment  changes  (better  lighting,  improved  contrast),  and  improved  human  detection 
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capabilities  (increasing  visual  lobe,  increasing  target  conspicuity).  At  the  same  time,  this  also  indicates 
training  as  a  critical  need  to  attain  satisfactory  sensory  performance. 

Tables  5.14  and  5.16  also  highlight  rule-based  behavior  resulting  in  the  identification  and  classification 
of  defects  as  a  significant  mode  of  visual  inspection.  Thus,  finding  corrosion,  wear,  small  cracks  and 
similar  difficult  defects  takes  place  because  of  rule-based  behavior.  It  is  pertinent  to  note  at  this  point  that 
the  work  card  system  used  in  the  aircraft  industry  to  control  aircraft  maintenance  and  inspection  relies 
heavily  on  a  linear  procedural  approach  (Drury,  1991;  Drury,  Prabhu  and  Gramopadhye,  1990). 
Rule-based  behavior  also  accounts  for  search  strategies  based  on  past  experience  and  work  card 
instructions.  Thus,  we  reach  the  conclusion  that  it  is  very  important  to  develop  procedural  knowledge 
(workcard  design),  checklists,  and  comparison  standards  to  support  this  behavior. 

Knowledge-based  behavior  is  often  a  slow  and  error-prone  process  and  creates  a  high  cognitive  workload 
for  the  human.  Often  in  such  circumstances  the  human  will  try  to  minimize  cognitive  strain  by  using 
shortcuts  in  the  reasoning  and  decision  making  processes,  which  can  lead  to  suboptimal  performance. 
Thus,  we  should  try  to  design  the  system  and  the  information  environment  to  minimize  the  need  to  indulge 
in  knowledge-based  behavior  Knowledge-based  behavior  in  visual  inspection  will  be  more  evident  in  a 
novice  inspector;  this  provides  a  strong  impetus  to  the  design  of  adequate  training  programs  to  bring  the 
novice  to  expert  levels  and  thus  minimize  knowledge-based  behavior.  Once  a  certain  level  of  expertise 
is  attained  the  knowledge-based  behavior  will  be  needed  only  in  case  of  unfamiliar  work  situations.  For 
example,  this  can  happen  if  an  inspector  who  normally  works  on  only  a  specific  part  of  the  aircraft  (e.g., 
the  wing  section)  is  asked  to  inspect  a  cargo  door.  Thus,  it  becomes  important  that  the  workcard 
(feedforward  environment)  be  designed  for  usability  and  have  the  information  needed  to  make  a  smooth 
transition  to  an  unfamiliar  task.  Feedback  information  from  a  buddy  system,  and  efficient  communication 
lines  with  the  supervisor,  also  have  to  be  considered.  Also  important  is  the  development  of  the  knowledge 
about  the  spatial  and  functional  aspects  of  the  aircraft,  which  is  partly  built  through  the  years  of  prior 
experience  of  the  inspector  as  an  aviation  mechanic.  This  is  normally  five  years  but  is  decreasing  due  to 
a  shortage  of  inspectors,  with  some  inspectors  having  as  little  as  three  years  of  maintenance  experience. 
There  are  cognitive  error  implications  in  too  rapid  a  promotion  system. 

5.3J.2.2  Non-Destructive  Inspection 

Moving  to  NDI  inspection,  skill-based  behavior  is  predominant  while  using  the  probe  and  is  a 
sensorimotor,  feedback-controlled  movement.  This  indicates  the  need  for  manual  control  training  on 
tracking  tasks  (e.g.,  circle  drawing,  tracking)  which  transfer  to  this  movement  control  task.  Similarly, 
thought  should  be  given  to  providing  tracing  paths  (e.g.,  circles  around  rivets)  which  provide  adequate 
feedback  information.  Templates  can  and  are  being  used  (although  some  inspectors  do  not  like  to  use 
them  due  to  handling  difficulties)  and  the  improved  design  and  use  of  such  aids  should  be  encouraged. 
The  rule-based  behavior  component  of  calibration  points  to  the  necessity  of  developing  adequate  and  well 
designed  checklists,  along  with  procedural  knowledge,  for  reliable  performance.  Swain  and  Weston  (1988) 
point  out  that  during  the  calibration  procedures,  powerplant  technicians  who  very  often  have  followed 
written  steps,  rely  on  memory  and  this  increases  the  probability  of  omissions.  This  points  to  a  calibration 
process  design  that  is  capable  of  providing  cues  to  the  next  step  on  the  display  screen  as  well  as  detecting 
wrong  inputs  by  the  operator.  Where  calibration  can  be  rigidly  defined,  the  checklist  is  the  obvious 
cognitive  aid,  already  extensively  used  in  aviation.  Those  calibration  tasks  which  have  some  flexibility 
must  be  clearly  delineated  for  separate  treatment. 
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Display  interpretation  forms  the  critical  portion  of  NDI  and  as  such  can  be  either  rule-based,  or 
knowledge-based,  or  both.  The  information  environment  should  thus  support  both  these  behaviors  white 
trying  to  ensure,  through  system  design  and  training,  that  the  need  for  knowledge-based  behavior  is 
minimized.  Since  rule-based  behavior  is  based  on  signs  which  trigger  stored  patterns  which  in  turn  control 
our  choices,  Rasmussen  and  Vicente  (1989)  suggest  that  the  design  of  the  display  should  be  such  as  to 
provide  action  cues  as  signs  which  also  have  symbolic  content,  thus  supporting  both  rule  and 
knowledge-based  performances.  Display  screens  for  NDI  that  allow  comparisons  of  the  current  pattern 
(curve)  with  known  defect  curves  for  comparative  decision  making  should  be  considered.  Also,  the 
knowledge-based  component  found  during  display  interpretation  indicates  the  need  to  develop  feedforward 
information  (training  and  documentation)  to  provide  technology  knowledge,  instrument  knowledge,  and 
aircraft  defect  history. 

It  must  be  emphasized  at  this  point  that  in  aircraft  inspection,  skill-based,  rule-based,  and  knowledge-based 
behaviors  are  not  necessarily  stand-alone,  discrete  behavior  modes.  Indeed,  they  overlap  on  some 
occasions  and  support  each  other  on  others.  For  example,  the  skill-based  behavior  of  probe  movement 
is  supported  by  either  knowledge-based  (for  the  novice)  or  rule-based  (for  the  expert)  behavior  that 
ascertains  the  boundaries  of  the  movement.  For  example,  the  probe  should  not  cut  the  livet  head  and  a 
movement  too  close  to  an  edge  should  be  avoided  since  both  of  these  will  show  defect  indications  without 
the  presence  of  any  defects.  Similarly,  rule-based  behavior  of  defect  identification  and  classification  in 
visual  inspection  is  sometimes  supported  by  knowledge-ba.sed  behavior  that  uses  active  reasoning  based 
on  a  deeper  and  functional  understanding  of  the  aircraft.  For  example,  during  visual  inspection  of  the 
wing  leading  edge,  the  inspector  who  is  looking  for  dents  may  reason  that  a  dent  forward  of  the  aileron 
trim  tab  may  be  more  important  than  one  in  another  area  because  it  could  cause  flow  breakup  in  an  area 
important  to  flight  control.  This  and  the  preceding  example  highlight  the  often  symbiotic  relationship  of 
the  different  behavior  modes.  Thus,  while  we  concentrate  on  skill-based  and  rule-based  behavior  of  the 
inspector  (since  these  are  the  dominant  behaviors),  we  also  need  to  understand  and  support  the 
knowledge-based  behavior  through  adequate  training  schemes,  documentation,  and  communications. 

From  the  discussion  above,  it  is  evident  that  the  mapping  of  the  inspection  processes  to  the  SRK 
framework  provides  useful  guidelines  for,  and  a  better  understanding  of,  the  type  of  information  that  has 
to  be  provided  for  aircraft  inspection.  This  has  been  compiled  in  Table  5.17  where  tlie  information 
categories  (feedforward  and  feedback)  identified  in  the  aircraft  inspection  information  model  (Figure  5.3) 
have  been  assigned  to  the  various  inspection  subtasks  based  on  the  type  of  behavior  they  would  logically 
support. 

5.3JJ  Analysis  of  Information  Requirements:  An  Error  Taxonomic  Approach 

In  an  analysis  of  93  major  accidents  for  a  24  year  period  from  1959  to  1983,  Sears  (1986)  found  that  I29r 
were  caused  by  maintenance  and  inspection  deficiencies.  Similarly,  Nagel  (1988)  reports  that 
approximately  four  out  of  every  hundred  accidents  that  occurred  in  the  worldwide  jet  fleet  from  1977  to 
1988,  had  maintenance  error  as  one  of  the  chief  causes.  As  shown  in  Section  5.3.2,  the  effects  of  human 
error  are  becoming  increasingly  unacceptable  and  the  issue  of  maintenance  and  inspection  error  is  being 
closely  examined  and  discu,sscd  in  the  aviation  community  (Drury,  1991). 

Formulation  of  information  environment  requirements  should  include  the  notion  of  human  error  and  its 
impact  on  aircraft  inspection.  Control  of  errors  to  an  acceptable  minimum  is  the  implicit  goal  of  all 
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human-machine  systems.  In  aircraft  inspection,  where  the  existence  of  certain  defects  in  an  aircraft  ready 
to  fly  is  almost  unacceptable,  it  is  pertinent  to  make  this  goal  explicit,  by  defining  information 
requirements  based  on  human  error  avoidance.  It  can  be  argued  that  information  provided  at  the  right 
time,  at  the  right  place,  in  the  right  manner,  is  at  least  a  necessary  condition  for  minimal  error 
performance. 


INSPECTION 

PROCESSES 

INFORMATION  ENVIRONMENT 

FEEDFORWARD 

FEEDBACK 

1.  VISUAL  (e.g.  Rivet 

Inspection) 

•  Pre-Attentive 

•  Training 

•  Foveal  Search 

•  Training 

•  Output  Feedback 

•  Foveal  Decision 

•  T  raining 

•  Procedural  Knowledge 

•  Comparison  Standards 

•  Cognitive  Feedback 

•  Buddy  System 

•  Extra-Foveal 

•  Knowledge  of  Cues 

•  Feedback  of  Results 

•  Decision  Making 

•  Co-Worker  Information 

•  Functional  System  Knowledge 

•  Fault  Causation  Knowledge 

•  Aircraft  History  (Defects) 

•  Communication  Links 

•  Buddy  System 

•  Cognitive  Feedback 

2.  NDI  (e.g.  Eddy  Current) 

•  Calibration 

•  Checklists,  Display  Design 

•  Probe  Movement 

•  Training  on  Tracking  and 

Accurate  Movement  Control 

•  Probing  Aids  (Templates 
or  Markings  Around 

Rivets) 

•  Display 

Interpretation 

•  Display  Design 

•  Functional  System  Knowledge 

•  Technical  Instrument  Knowledge 

•  Aircraft  History 

•  Cognitive  Feedback 

Table  5.17  Information  Requirements  Identified  from  Mapping  Inspection  Processes  to  SRK 
Framework  for  Two  Examples 


5.3JJ.1  Methodology  for  Information  Requirement  Formulation 

Human  error  can  serve  as  an  effective  platform  to  study  and  formulate  the  information  requirements  of 
aircraft  inspection  just  as  it  was  used  in  Section  5.3.2  to  understand  the  overall  inspection  process.  We 
present  below  a  methodology  that  attempts  to  guide  the  design  of  the  information  environment  to 
controlling  human  error; 
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1.  Identify  and  define  tlic  levels  of  the  system  under  consideration  (e  g.,  management.  supx;rvisory. 
lead  inspector,  inspector). 

2.  At  the  level  under  analysis,  define  the  functional  requirement  of  the  level,  current  allocation  of 
human-computer  functions,  and  interactions  with  the  other  levels. 

3.  Develop  a  human  error  taxonomy  for  the  level  under  consideration. 

4.  Use  the  taxonomy  and  the  functions  identified  in  step  2  to  outline  the  failure  modes  (phenotypes) 
and  associated  mechanisms  of  human  malfunction  and  error  shaping  factors  (geno-types)  specific 
to  each  function. 

5.  Identify  the  component  of  the  information  system  that  would  be  necessary  to  cr)ntrol  human  error 
based  on  understanding  of  the  phenotypes  and  genotypes  of  .step  4. 

6.  Define  the  requirements  of  each  information  component:  (I)  what  information  to  present 
(information  quality);  (2)  when  to  present  such  information  (information  flow);  and  (3)  how  to 
pre.sent  this  information  (information  display),  so  that  the  human  error  potential  is  minimized. 

The  above  methodology  combines  a  task  analytic  approach  with  a  human  error  taxonomy  so  that 
information  requirements  are  formulated  to  control  human  error.  Obviously,  the  error  taxonomy 
development  is  an  impmiani  pan  of  this  approach.  A  framework  or  guideline  is  presented,  which  can  be 
used  to  develop  a  taxonomy  for  use  in  this  methodology. 

Rasmussen  and  Vicente  (1990)  suggest  that  human  error  analysis  can  be  performed  from  two  different 
perspectives.  The  first  perspective  tries  to  identify  possible  human  errors  and  their  effects  on  sy,stcm 
performance,  while  the  second  perspective  aims  at  improving  system  design  to  eliminate  the  effects 
identified  in  the  analysis  from  the  first  perspective.  Ba.sed  on  the  fir.st  perspective.  Drury  (1991)  developed 
an  error  taxonomy  from  the  failure  modes  of  each  task  in  aircraft  inspection.  This  taxonomy  has  been 
developed  based  on  the  recognition  that  a  pro-active  approach  to  error  control  is  needed  to  identify 
potential  errors.  Thus,  the  taxonomy  is  aimed  at  the  phenotypes  of  error  (Hollnagel,  19X9),  i.e.,  the  way 
errors  are  observed  or  appear  in  practice.  In  Section  5.3.2  it  was  also  noted  that  Rasmussen  and  Vicente 
(1990)  propose  a  taxonomy  from  the  viewp<iint  of  identifying  po.ssible  improvements  in  system  design 
with  categories  of  errors  as  related  to;  (a)  effects  of  learning  and  adaptation,  (b)  interference  among 
competing  control  structures,  (c)  lack  of  resources,  and  (d)  stochastic  variability.  They  suggest  that 
different  methods  have  to  be  adopted  to  control  the  errors  associated  with  each  of  the  abtwe  four 
categories,  and  that  it  is  necessary  to  make  the  system  enror-tolerant  to  achieve  reliable  system 
performance. 

We  propose  that  the  failure  modes  identified  in  the  taxonomy  of  aircraft  in.spection  by  Drury  (1991)  can 
be  classified  using  the  systemic  error  mechanisms  categories  and  the  cognitive  control  categories  proposed 
by  Rasmussen  and  Vicente  (1989).  (An  example  is  given  in  Table  5,18.  for  error  modes  in  llie  decision 
task.)  In  Table  5.19,  such  an  assignment  is  .shown  u.sing  the  failure  modes  for  the  decision  task.  For  each 
behavior  mode  (i.e,  skill,  rule,  or  knowledge)  the  genotypes  of  errors  can  be  then  po.stulated.  Genotypes 
are  the  contributing  psychological  cau.ses  of  errors  and  are  repre.sent alive  of  the  characteristics  of  the 
human  cognitive  sy.stem  (Hollnagel,  1989).  Table  5.20  shows  the  genotypes  a.ssigned  to  the  different 
behavior  modes. 
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TASK 

ERROR(S) 

TASK  4  -  DECISION 

4.1  Interpret  indication 

4.1.1  Classify  as  wrong  fault  type. 

4.2  Access  measuring  equipment. 

4.2.1  Choose  wrong  measuring  equipment. 

4.2.2  Measuring  equipment  is  not  available. 

4.2.3  Measuring  equipment  is  not  working. 

4.2.4  Measuring  equipment  is  not  calibrated. 

4.2.5  Measuring  equipment  has  wrong  calibration. 

4.2.6  Does  not  use  measuring  equipment. 

4.3  Access  companson  standards. 

4.3.1  Choose  wrong  comparison  standard. 

4.3.2  Comparison  standard  is  not  available. 

4.3.3  Comparison  standard  is  not  correct. 

4.3.4  Comparison  standard  is  incomplete. 

4.3.5  Does  not  use  comparison  standard. 

4.4  Decide  on  fault  presence. 

4.4.1  Type  1  error,  false  alamn. 

4.4.2  Type  2  error,  missed  fault. 

4.5  Decide  on  action. 

4.5.1  Choose  wrong  action. 

4.5.2  Second  opinion  if  not  needed. 

4.5.3  No  second  opinion  if  needed. 

4.5.4  Call  for  buy-back  when  not  required. 

4.5.5  Fail  to  call  for  required  buy-back. 

4.6  Remember  decision/action. 

4.6.1  Forget  decision/action. 

4.6.2  Fail  to  record  decision/'action. 

OUTCOME  4:  All  indications  located  are  correctly  classified,  correctly  labelled  as  fault  or  no  fault,  and  actions 
correctly  planned  for  each  indication. 

Table  5.18  Task  and  Error  Taxonomy  lor  Inspection,  Task  4,  Decision 


Tlie  above  (ramework,  then,  allows  the  opportunity  to  examine  each  failure  mode  within  the  context  of 
(a)  the  cognitive  behavior  from  which  it  results,  (b)  the  systemic  error  category  in  which  it  occurs,  and 
(c)  the  internal  error  mechanisms  that  are  the  probable  causes  of  these  malfunctions.  An  analysis  of  this 
information  can  then  form  (he  basis  of  system  design  to  minimize  or  eliminate  the  failure  nuxles.  From 
the  information  requirements  viewpoint,  system  design  considerations  should  Ihen  drive  the  specifications 
as  to  the  type,  location,  and  temporal  position  of  the  information.  Preliminary  recommendations  on  the 
type  of  information  component  have  been  li.sted  in  Table  5,20.  In  actual  use.  Table  5.20  should  be  utilized 
as  a  framework  for  an  error  taxonomy  which  can  be  applied  in  the  task  analysis  methodology  proposed. 

5.3J.4  Testing  the  Information  Framework 


Using  the  in.spection  program  developed  lor  NDI  (Section  5.3. 1.1)  it  is  possible  to  make  direct 
experimental  tests  of  many  of  the  predictions  coming  from  the  framework  being  developed  in  Sections 
5.3J.2  and  5J.3J.  As  a  demonstration  of  the  u.se  of  the  NDI  in.spection  program,  a  relatively  simple 
experiment  ba.sed  on  the  information  requirements  was  conducted.  It  involved  training  two  groups  of 
subjects  on  the  in.spection  task,  then  either  providing  or  not  providing  off-line  feedback  of  performance, 
and  finally  measuring  inspection  performance  of  both  groups. 
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SYSTEMIC 

ERROR 

BEHAVIOR  CATEGORIES 

SKILL-BASED 

BEHAVIOR 

RULE-BASED 

BEHAVIOR 

KNOWLEDGE-BASED 

BEHAVIOR 

1.  Effects  of 
Learning 
and 

Adaptation 

4.2.1,  4.2.4,  4.2.6, 

4.3.1,  4.3.5,  4.5.1, 

4.6.2 

4.2.4  Fail  to  calibrate  meas.  eqpt. 
4.2.6  Fail  to  use  meas.  eqpt. 

4.3.5  Fail  to  use  comp.  std. 

4.6.2  Fail  to  record  decision. 

4.2.1  Choose  wrong  meas.  eqpt. 

4.3.1  Choose  wrong  comp.  std. 

4.5.1  Choose  wrong  action. 

4.2.5  Meas.  eqpt.  wrongly  calib. 

4.1.1  Classify  as  wrong  fault. 

4.4.1  False  ^arm. 

4.4.2  Missed  fault. 

4.1.1  Classify  as  wrong  fault. 

4.4.1  False  alamri. 

4.4.2  Missed  fault. 

4.5.2  Wrong  decision  on  getting. 

4.5.5  Second  opinion. 

4.5.4,  4.5.5  Wrong  decision  on 

calling  for  buyback. 

4.2.5  Wrong  calibration  of 
meas.  eqpt. 

2.  Interference 
Among 
Competing 
Control 
Structures 

4.2.5 

4.1.1,  4.4.4,  4.4.2 

4.5.1 

4.5.2,  4.5.3,  4.5.4,  4.5.5 

4.1.1,  4.4.1,  4.4.2,  4.5.2,  4.5.3,  4.5.4, 
4.5.5 

3.  Lack  of 
Resources 

4.6.1  Forget  decision 

4.5.2,  4.5.3 

4.5.4,  4.5.5 

4. Stochastic 
Variability 

4.1.1,  4.4.1,  4.4.2 

4.6.1 

Table  5.19  Assignment  of  Systemic  Error  Mechanisms  to  Failure  Based  on  Behavior  Types  for  the 
Decision  Making  Component  of  Aircraft  Inspection 


As  shown  in  Section  5J.3.1,  on-the-job  feedback  can  be  a  powerful  performance  enhancer,  but  it  is  an 
expensive  one  to  implement.  It  involves  re-inspection  of  an  inspector’s  work  by  a  (presumably  more 
reliable)  auditor,  a  process  which  adds  cost  in  proportion  to  the  percentage  of  work  audited.  A  more 
realistic  approach  would  be  to  provide  feedback,  for  exampld  by  having  the  inspector  inspect  a  test  piece 
with  a  known  set  of  faults,  between  regular  inspection  tasks.  Feedback  can  easily  be  provided  from  such 
a  test  piece,  but  we  need  to  measure  the  effectiveness  of  such  feedback.  A  test  of  this  effectiveness  also 
provided  a  useful  practical  test  of  the  NDl  program,  and  indeed  many  pilot  subjects  were  run  and  program 
modifications  were  made  before  the  complete  experiment  reported  here  was  started.  The  following  is  a 
brief  description  of  the  experiment  and  its  results.  These  results  are  being  presented  in  more  detail  in  a 
separate  project  report. 

5.3J.4.1  Methodology 


Two  groups  of  eight  subjects  each  were  chosen  randomly  from  a  population  replying  to  advertisements. 
All  were  currently  unemployed  members  of  the  work  force,  with  males  and  females  and  a  variety  of  ages 
represented.  Each  subject  was  given  two  pre-tests,  both  of  which  had  been  shown  to  correlate  with 
performance  on  industrial  inspection  tasks.  The  first  was  the  Embedded  Figures  Test  (EFT)  which 
cla.ssifies  the  cognitive  style  of  a  person  as  Field  Dependent  (i.e.,  highly  influenced  by  die  visual  context 
of  a  task)  and  Field  Independent  (i.e.,  more  able  to  cognitively  restructure  a  task  independent  of  its  visual 
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context).  The  second  test  was  the  Matching  Familiar  Figures  Test  (MFFT)  which  measures  the  tendency 
of  subjects  to  opt  for  speed  or  accuracy  in  their  speed/accuracy  tradeoff  (see  Section  5J.4).  Foveal  visual 
acuity  was  also  measured. 

Both  groups  were  given  the  same  training  in  the  principles  of  eddy-current  inspection  of  rivets  for  cracks, 
in  controlling  of  the  pointer  using  the  mouse,  and  in  interpreting  the  meter  needle  movements.  This 
training  occupied  about  four  hours.  Following  training,  the  eight  subjects  in  the  control  group  were  tested 
on  a  task  involving  420  rivets  followed  by  a  task  involving  80  rivets,  on  each  of  four  days.  The 
experimental  group  was  given  the  same  task  except  that  they  were  provided  with  feedback  on  the  missed 
cracks,  false  alarms,  and  performance  time  on  the  80  rivet  task. 

In  the  main  task,  the  same  measures  of  misses,  false  alarms,  and  task  time  were  taken  for  each  subject. 
5.3J.4.2  Results 

Analyses  of  covariance  were  performed  on  the  measures  of  total  time,  misses,  false  alarms,  and  derived 
measures  from  Signal  Detection  Theory  (Section  5.3,5)  of  sensitivity  (d’)  and  criterion  (XJ.  Each  analysis 
tested  for  differences  between  the  two  groups  (G),  for  differences  between  the  four  days  (D),  as  well  as 
for  their  interaction  (D  X  G).  Two  sets  of  covariates  were  derived  from  factor  analysis  to  contain  the 
following  components: 

Covariate  1 :  EFT  Errors,  EFT  Times.  MFFT  Errors 

Co  variate  2;  MFFT  Time  (negative).  Visual  Acuity 

Covariate  1  represents  poor  accuracy  performance  and  field  dependence,  while  Covariate  2  represents  fast 
performance  with  good  vision.  Table  5.21  summarizes  the  analyses  of  covariance  of  the  measures  taken. 
There  were  no  significant  group  effects,  and  only  a  single  day  effect,  that  on  total  lime  for  the  task 
Covariate  2  was  significant  for  total  time  and  for  criterion  X^.  Hgure  5.8  shows  plots  of  the  results  for 
times,  misses,  false  alarms,  and  sensitivity  (d’)  comparing  the  experimental  and  control  groups  across  the 
four  days  of  the  experiment. 


Measured  Artalyzed 

Groups  (G) 

Days  (D) 

G  XO 

Covariate  1 

Covariate  2 

Total  Time 

P  <  0.0001 

— 

_ 

P  s  0.0235 

Misses 

— 

— 

— 

— 

False  Alarms 

— 

— 

— 

— 

* 

Sensitivity  (d’) 

— 

— 

— 

— 

— 

Criterion  (XJ 

— 

— 

— 

— 

P  S  0.0355 

*  Indicates  that  the  Visual  Acuity  component  of  covanate  2  was  significant  at  P  <  0.0244. 


Table  5.21  Summary  of  Analyses  of  Covariance  for  Off-Line  Feedback  Experiment 
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5.3J.4.3  Discussion 

The  major  finding  of  this  first  experiment  using  the  NDl  program  was  one  of  very  high  between-subject 
and  day-to-day  variability.  The  between-subject  variability  was  expected,  and  it  appears  that  some  of  this 
variability  at  least  is  predictable  using  the  covariates  derived.  Because  of  this  variability,  the  effects 
shown  in  Figure  5.8  do  not  reach  statistical  significance  with  only  eight  subjects  per  group.  Having  said 
this,  there  is  an  indication  in  all  four  parts  of  Figure  5.8  that  the  experimental  group  outperforms  the 
control  group  by  the  end  of  the  experiment. 

Over  the  four  days  of  the  experiment,  accuracy  performance,  as  measured  by  misses  and  false  alarms, 
improved  slightly  for  the  control  group  and  somewhat  more  for  the  experimental  group.  Despite  this 
overall  improvement,  the  day-to-day  improvement  was  erratic.  Total  time  decreased  for  both  groups,  with 
the  experimental  group  being  more  rapid  than  the  control  group  throughout.  Sensitivity,  as  defined  in 
Signal  Detection  Theory,  marginally  favored  the  control  group  until  the  final  day,  when  the  experimental 
group  continued  to  improve  while  the  control  group  regressed  slightly. 

During  the  course  of  the  experiment,  it  was  clear  that  the  experimental  group  was  using  the  off-line 
feedback  to  modify  their  inspection  strategy.  However,  this  process  involved  trial  and  error,  which  gave 
considerable  variability  of  performance.  The  performance  feedback  helped  somewhat,  but  would  have 
been  much  easier  to  interpret  if  it  had  contained  hints  and  steps  that  the  inspectors  could  take  to  make  the 
improvements  they  knew  were  needed.  Cognitive  feedback,  as  postulated  in  Section  5J.3.1  appears  to 
be  required  if  inspectors  are  to  make  use  of  their  own  performance  data. 

5.3J.4.4  Conclusions 

While  off-line  performance  feedback  was  marginally  effective,  the  high  variability  between  subjects 
prevented  significant  results  from  being  obtained.  At  least  part  of  the  day-to-day  variability  was  due  to 
subjects  using  the  feedback  in  an  unguided  manner  in  an  attempt  to  improve,  suggesting  that  cognitive 
feedback  may  be  needed  to  supplement  off-line  performance  feedback.  The  small  size  of  the  feedback 
task  (80  rivets)  might  also  have  failed  to  provide  sufficient  data  to  significantly  aid  in  transfer  of  feedback 
results.  The  significant  covariates  for  total  time  and  criterion  also  indicate  influence  of  other  independent 
factors,  namely  visual  acuity  and  cognitive  style. 

5.3.4  A  FRAMEWORK  FOR  SPEED/ACCURACY  TRADEOFF  IN  AIRCRAFT  INSPECTION 

In  almost  any  discussion  with  aircraft  maintenance  personnel,  maintenance  managers,  regulatory  bodies, 
or  the  traveling  public,  the  general  issue  of  inspection  accuracy  arises.  More  specifically,  in  the 
post-deregulation  environment  of  U.S.  commercial  aviation,  the  effect  of  time  pressures  on  the  inspection 
system  (particularly  the  human  inspector)  is  causing  concern.  This  section  reviews  the  functions  and  tasks 
of  aircraft  inspection,  based  upon  a  two-year  observational  .study  of  the  system,  and  uses  prior  studies  of 
human  inspection  to  examine  the  possibilities  of  time  pressure  affecting  accuracy.  A  Speed/Accuracy 
Tradeoff  (SATO)  perspective  is  taken,  i.e.  how  do  speed  and  accuracy  co-vary  in  inspection. 

Both  speed  and  accuracy  are  relatively  easy  to  define  in  inspection. 
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Speed:  The  rate  of  inspecting  items,  usually  measured  as  the  reciprocal  of  the  time(  t  ) 

taken  to  inspect  a  single  item  or  defined  area. 

Accuracy:  False  Alarm  (Type  1  error)  The  probability  of  an  inspector  responding  that  a 

defect  exists,  when  in  truth  it  does  not. 

Miss  (Type  2  error)  The  probability  of  an  inspector  failing  to  respond  that  a 
defect  exists,  when  in  truth  it  does  exist. 

This  section  is  concerned  explicitly  with  the  co-variation  of  (  f  ) ,  False  Alarms,  and  Misses. 

From  an  airline  management  perspective,  two  goals  need  to  be  achieved  by  the  system:  safety  and 
profitability.  The  profitability  goal  can  only  be  achieved  by  first  ensuring  that  the  safety  goal  is  achieved 
economically.  These  objectives  are  passed  through  sometimes  complex  organizational  systems  (Taylor, 
1990)  to  supervisors  and  finally  to  inspectors.  At  the  inspector’s  level  two  goals  need  to  be  achieved  by 
the  inspection  system:  accuracy  and  speed.  Accuracy  means  detecting  those  indications  (faults)  which 
must  be  remedied  for  the  safe  operation  of  the  aircraft  while  not  activating  the  maintenance  system  for 
non-faults.  Speed  means  the  task  must  be  performed  in  a  timely  manner  without  the  utilization  of 
excessive  resources.  These  two  criteria  of  the  inspection  system  can  be  expected  to  be  inversely  related 
at  the  inspection  level  (Drury,  1985). 

When  inspection  is  split  into  its  task  steps  (Table  5.1  of  Section  5.1.1),  it  can  be  seen  that  all  of  the  tasks 
require  both  speed  and  accuracy  for  their  completion.  However,  the  most  error-prone  activities  in 
industrial  inspection  are  the  search  and  decision  making  tasks  (Drury,  1984)  while  access  is  an  activity 
whose  time  must  be  minimized  for  efficient  operation. 

The  speed  and  accuracy  with  which  each  of  the  components  is  performed  depends  upon  the  relative 
utilities  of  the  various  outcomes  to  the  inspectors.  Utility  is  a  concept  that  can  be  used  in  models  of  the 
inspector  as  a  maximizer  or  optimizer  (Drury,  1992)  to  give  a  normative  model  as  a  starting  point  for 
more  realistic  inspector  models.  Thus,  the  optimum  speed  and  accuracy  is  not  defined  in  terms  of 
minimizing  or  maximizing  one  particular  aspect  of  inspection  but  is  defined  in  terms  of  a  performance 
which  yields  the  highest  overall  utility. 

If  a  task  can  be  performed  at  various  levels  of  speed  and  accuracy,  then  it  is  possible  (Wickens,  1984)  to 
generate  an  operating  characteristic  curve  (see  Figure  5.9a)  relating  the  two  measures.  Any  point  on  the 
speed/accuracy  operating  characteristic  (SAOC)  curve  shows  the  accuracy  with  which  the  task  can  be 
performed  at  a  particular  speed.  Hence  to  meet  the  designed  system  objectives  of  speed  and  accuracy, 
it  is  essential  that  the  inspectors  operate  at  the  correct  point  on  the  correct  operating-characteristic  curve. 

In  order  for  an  inspector  to  choose  a  particular  strategy  from  the  set  of  available  strategies,  it  is  necessary 
to  determine  the  utility  of  all  the  candidate  strategies,  as  a  function  of  speed  and  accuracy.  The  utility  can 
be  computed  for  every  point  in  the  joint  performance  space  (Speed,  Accuracy),  whether  that  point  is 
achievable  or  not.  Now  by  knowing  the  utility  function  it  is  possible  to  determine  the  optimal  operating 
point,  i.e.  that  which  maximizes  the  expected  utility.  Typically,  contours  of  equal  utility  are  superimposed 
upon  the  SAOC  to  show  where  this  optimal  operating  point  occurs  (see  Figure  5.9b).  This  section 
considers  access,  .search,  and  decision  making  in  turn,  and  uses  models  of  each  to  .show  the  form  of  the 
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SAOC.  Models  are  not  developed  in  detail.  For  more  information  the  original  report  (Drury  and 
Gramopadhye,  1991)  can  be  consulted.  Each  model  is  an  optimization  model,  showing  how  an  inspector 
may  be  expected  to  choose  between  alternative  strategies.  In  large  decision  tasks,  there  is  considerable 
evidence  that  they  are  satisfiers  rather  than  maximizers  (Wickens,  1991).  However,  in  small  task 
components  such  as  those  found  in  inspection,  optimization  models  represent  a  good  starting  point  for 
consideration  of  the  factors  involved  (Drury,  1988;  Chi,  1990). 


(SAOC) 

5.3.4.1  Factors  Affecting  Access  Tasks 

The  access  task  consists  of  physically  reaching  the  area  to  be  inspected.  This  may  be  an  unaided  human 
task  (e.g.,  area  insp)ection  of  lower  fuselage  skin),  aided  by  access  devices  (e.g.,  steps,  scaffolding, 
cherrypickers),  or  require  access  through  intervening  structure  (e.g.,  inspection  of  interiors  of  wing  fuel 
tanks  through  access  holes).  All  of  these  activities  involve  connolling  the  movement  of  the  inspector’s 
body,  or  body  parts,  within  a  restricted  space.  In  general,  control  theoretic  models  of  the  human  operator 
in  control  tasks  (Sheridan  and  Ferrell,  1974;  Wickens,  1984)  show  that  as  more  speed  is  demanded, 
tracking  accuracy  decreases-a  typical  SATO.  If  the  access  task  is  modelled  as  moving  accurately  between 
two  boundaries  without  making  an  error  of  exceeding  a  boundary,  then  the  self-paced  tracking  models  of 
Drury  (1971)  and  Montazer,  Drury  and  Karwan  (1987)  can  apply.  Thus,  moving  the  body  along  the 
walkway  of  a  scaffold  without  hitting  (and  possibly  damaging)  the  aircraft  structure  on  one  side  or  the 
scaffold  rail  on  the  other  is  such  a  task.  Moving  the  hand  (or  head)  through  an  access  hole  or  moving 
a  cherrypicker  along  the  fuselage  upper  skin,  (although  only  one  physical  boundary  exists  here)  are  further 
examples. 

The  self-paced  tracking  model  considers  the  inspector,  or  a  vehicle  controlled  by  the  inspector,  as  choosing 
a  speed  which  will  maximize  the  utility  to  the  iaspector.  Utility  is  compo.sed  of  rewards  for  speed  and 
penalties  for  error,  in  this  case  the  error  of  exceeding  the  fixed  boundaries.  Model  results,  and 
experimental  data  from  a  variety  of  studies,  have  shown  that  the  speed  cho.sen  increa.ses  with  space 
available  (e.g.,  width)  until  some  limiting  speed  is  reached.  The  three  factors  affecting  performance  are 
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thus  space  available,  the  ease  of  control  (controllability)  of  the  vehicle,  and  the  inspector’s  perception  of 
the  relative  utilities  of  speed  and  accuracy.  Each  will  be  considered  in  turn. 

5.3.4.1.1  Space  Available 

Space  available  can  be  controlled  relatively  easily  around  the  aircraft,  but  access  within  the  airframe  itself 
is  largely  determined  at  the  design  stage.  With  older  aircraft  there  has  been  a  history  of  unpleasant 
surprises  for  maintenance  personnel  when  they  reached  service,  but  manufacturers  are  now  using 
computer-manipulable  human  anthropomorphic  models  (e.g.,  CREWCHIEF,  SAMMIE)  to  examine  access 
for  maintenance  before  structures  are  finalized.  Note,  however,  that  the  SATO  model  shows  that  more 
space  improves  performance,  so  that  the  minimum  necessary  for  physical  access  (e.g.,  for  a  95th  percentile 
male)  will  not  provide  optimum  performance.  A  human  anthropomorphic  model  only  gives  the  space 
required  for  a  person  to  statistically  assume  a  posture.  For  movement  (the  essence  of  maintenance  and 
inspection)  more  access  room  is  required  beyond  this  minimum.  The  same  considerations  apply  to  access 
around  the  aircraft.  Steps  and  walkways  should  be  made  wide  enough  to  provide  unhindered  movement, 
not  just  wide  enough  to  accommodate  a  large  static  human.  As  an  example,  Drury  (1985)  reports  that  for 
movement  through  a  doorway  both  performance  time  and  errors  decrease  from  the  anthropometric 
minimum  width  of  about  20  inches  to  the  unhindered  width  of  36  inches.  Very  similar  findings  are  used 
in  the  aviation  industry  to  determine  sizes  of  emergency  doors  for  passengers. 

During  Phase  III  of  this  work,  explicit  models  and  experiments  will  be  developed  to  test  the  effects  of 
space  available,  and  human  posture,  on  performance  and  stress  in  inspection  and  maintenance  activities. 

5.3.4. 1.2  Controllability 

Controllability  of  the  system  having  access  is  a  major  determinant  of  access  performance.  For  most  tasks, 
the  "system"  is  the  inspector’s  own  body,  the  most  naturally  controllable  system.  However,  controllability 
can  be  adversely  affected  by  equipment  carried  (flashlight,  tools,  work  cards,  NDI  equipment)  and  by  the 
quality  of  clothing  worn.  Thus,  coveralls  and  shoes  should  be  minimally  restrictive.  Shoes  should  also 
provide  good  grip  on  a  variety  of  surfaces  under  both  wet  and  dry  environmental  conditions. 
Controllability  will  be  decreased  by  any  impairment  of  the  human,  for  example  sickness,  alcohol,  or  drugs, 
reinforcing  the  control  required  over  such  conditions  at  the  work  place. 

For  control  of  systems  such  as  vehicles  (e.g.,  cherrypickers,  wheeled  steps,  moveable  access  scaffolds)  a 
considerable  body  of  information  exists  (e.g.,  Wickens,  1984)  on  the  human  as  controller.  Most  of  these 
recommendations  apply  equally  to  the  self-paced  access  tasks  considered  here.  Thus  for  example,  controls 
should  move  in  the  same  directions  and  sense  as  the  element  they  control.  It  should  be  noted  that  many 
cherrypickers  have  hydraulic  or  electrical  controls  which  violate  this  principle.  Direction  of  motion  errors 
are  to  be  expected  with  such  systems,  causing  at  best  a  slowing  of  the  task  and  at  worse  damage  to  the 
aircraft  structure,  depending  upon  the  operator’s  SATO  choice.  These  same  controls  are  often  not 
progressive  in  operation,  but  "bang-bang"  controls,  either  fully  on  or  fully  off.  With  such  a  degraded 
control  system,  any  designed  speed  setting  is  a  poor  compromi.se.  At  times  it  is  too  slow,  causing  delay 
and  frustration  in  making  long  movements,  while  at  other  times  it  is  too  rapid,  causing  errors  and 
time-consuming  multiple  corrections  in  making  the  final  accurate  positioning  movements.  In  addition,  any 
time  lags  or  inertia  in  the  system  controlled  will  have  a  negative  impact  on  controllability. 
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Within  the  maintenance  hangar,  there  are  other  constraints  on  design  (or  choice)  of  access  equipment. 
Any  equipment  must  be  available  if  it  is  not  to  cause  delays,  suggesting  both  that  a  sufficient  supply 
exists,  and  that  it  is  well-scheduled.  The  difficulty  with  maintaining  a  sufficient  supply  is  that  such 
equipment  is  both  expensive  and  space-consuming.  The  typical  management  response  is  to  have  a  mixture 
of  special-purpose  equipment,  such  as  empennage  access  scaffolding,  and  standardized,  flexible  equipment, 
such  as  stepladders,  cherrypickers,  and  standard  moveable  platforms.  When  only  a  single  aircraft  type  is 
to  be  serviced,  as  in  most  large  airlines  and  specialist  repair  centers,  purpose-built  equipment  should,  and 
does,  predominate.  In  more  general  purpose  organizations,  the  emphasis  is  on  standardized,  flexible 
equipment.  However,  there  are  still  times  when  schedules  demand  more  access  equipment  than  is 
instantaneously  available.  It  is  at  these  times  that  available  equipment  is  substituted  for  correct  equipment 
to  avoid  delays.  The  result  is  lower  system  controllability,  with  the  potential  for  errors  affecting  both  job 
performance  and  personnel  satcty. 

5.3.4.1.3  Perception  of  Utilities 

Given  the  space  available  and  the  controllability  of  the  system,  the  balance  between  speed  and  accuracy 
is  still  finally  chosen  by  the  operator’s  own  SATO.  As  discussed  earlier,  this  is  where  any  gate  pressures 
or  schedule  demands  can  have  an  effect.  As  access  is  a  task  of  inspection  which  appears  non-critical,  it 
can  be  one  where  time  is  saved  for  tasks  perceived  as  more  important.  In  addition,  access  is  where 
pressures  from  other  members  of  the  maintenance  team  can  be  acute.  Co-workers  will  at  times  need  the 
access  equipment  the  inspector  is  using  or  vice  versa,  leading  to  time  pressures  over  a  short  time  scale 
even  where  none  exist  on  the  longer-term  scale  of  a  whole  maintenance  visit. 

Inspectors’  errors  in  access  are  defined  as  reaching  or  exceeding  the  boundary  of  available  space.  They 
thus  include  both  damage  to  aircraft  structure,  and  injury  to  the  inspector.  Humans  are  likely  to 
misperceive  the  risks  associated  with  such  rare  events,  both  in  terms  of  the  consequences  and  probabilities 
involved.  Particularly  with  highly  experienced  personnel,  such  as  insp)ectors,  the  probabilities  of  error  are 
typically  rated  lower  than  their  objective  values.  This  can  be  expected  to  lead  to  a  choice  of  SATO 
strategy  favoring  speed  rather  than  accuracy. 

5.3.4.2  Factors  Affecting  Search  Tasks 

The  process  of  visual  search  of  an  extended  area,  such  as  the  area  called  out  on  a  workcard,  has  been 
successfully  modeled  since  the  start  of  human  factors  engineering.  A  human  searcher  (e.g.,  the  inspector) 
makes  a  sequence  of  fixations,  centered  on  different  points  in  the  area.  During  a  fixation,  which  typically 
lasts  0.25-0.5  seconds,  the  inspector  can  detect  defects  in  an  area,  called  the  visual  lobe,  around  the 
fixalicr.  center.  Between  fixations  the  eye  moves  very  rapidly  and  can  take  in  very  little  information.  The 
sequence  of  fixations  can  either  be  modelled  as  random  (e.g.,  Krendel  and  Wodinski,  1960)  or  systematic 
with  repeated  scans  (e.g.,  Williams,  1966).  For  both  of  these  models,  equations  can  be  developed  relating 
the  probability  of  detection  to  the  time  spent  searching  (Morawski,  Drury  and  Karwan,  1980). 

In  general,  the  longer  an  inspector  searches  an  area,  the  greater  the  probability  of  a  target  being  located, 
with  dimini.shing  returns  as  search  time  is  increased.  Such  curves  are  the  SAOC’s  of  visual  search,  and 
are  shown  in  Figure  5.10.  Given  such  SAOC’s,  then  the  optimum  time  for  .searching  can  be  calculated 
(Morawski,  Drury  and  Karwan,  1992)  based  upon  the  reward  for  speed  and  the  penalty  for  error. 
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From  the  visual  search  models,  three 
groups  of  factors  determine  search 
performance: 

1.  Factors  affecting  the  visual 

lobe. 

2.  Factors  affecting  the  search 

strategy. 

3.  Factors  affecting  the  SATO 

and  stopping  policy. 

Based  on  the  defect  type,  severity 

level,  and  location,  the  defects  can  be 
classified  into  critical  and  non-critical 
defects.  Critical  defects  are  those 
defects  which  affect  the  airworthiness 
of  the  aircraft,  hence  whose  detection 
is  critically  important.  Non-critical 
defects  do  not  immediately  affect  the 
airworthiness  of  the  aircraft  but  have  to 
be  detected  in  the  long  run.  There  is 
clearly  a  heavy  penalty  for  missing 
critical  defects,  but  the  entire  area 
needs  to  be  searched  for  both  critical 
and  noncritical  defects  within  a 
specified  time  period.  Thus,  two  goals  need  to  be  achieved  by  the  inspector,  speed  and  accuracy,  for 
which  the  inspector  needs  to  be  efficient  as  well  as  effective.  In  order  to  understand  the  Speed/Accuracy 
Tradeoffs  in  search  where  the  inspector  is  looking  for  multiple  defect  types,  the  factors  which  affect  this 
tradeoff  and  indeed  the  whole  search  process  must  be  examined. 

5.3.4.2.1  Visual  Lobe  Factors 

According  to  Engel  (1971)  fault  conspicuity  is  defined  as  that  combination  of  properties  of  a  visual  object 
in  its  background  by  which  it  attracts  attention  via  the  visual  system,  and  is  seen  as  a  consequence.  Monk 
and  Brown  (1975)  have  shown  that  mean  search  times  increase  as  a  function  of  the  number  of  non-targets 
in  the  target  surroundings.  They  have  also  shown  that  isolated  targets  are  more  easily  detected  than  those 
surrounded  by  non-targets.  Williams  (1966)  has  shown  that  the  color  and  size  of  the  targets  can  be  used 
by  subjects  to  direct  their  eye  movements.  Studies  of  information  processing  within  a  single  fixation  have 
shown  that  the  probability  of  target  detection  increa.scs  with  increased  target  size  and  brightness  contrast, 
and  decreases  with  angular  distance  from  the  fixation  point  (Overington,  1973).  This  decrease  with 
off-axis  angle  provides  the  basis  for  determining  visual  lobe  size,  i.e.,  the  area  within  which  a  target  may 
be  detected  (Bloomfield,  1975). 

In  aviation,  this  search  performance  has  been  extensively  studied  and  modeled  to  determine  human 
performance  in  detection  of  military  targets  (for  example,  ground  targets  or  hostile  aircraft).  In  terms  of 


p(detect  before  time  t) 


Figure  5.10  Typical  Cumulative  Search  Time  Distribution, 
Giving  the  SAOC  for  Visual  Search 
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aviation  maintenance  inspection,  the  implication  is  that  lighting  and  other  target/background  amplification 
devices  should  be  used  to  make  the  conspicuity  of  a  defect  as  high  as  possible,  and  hence  increase  visual 
lobe  size. 

There  are,  of  course,  individual  differences  in  visual  lobe  size.  Eye  movement  studies  have  shown  that 
subjects  who  have  larger  visual  lobes  are  more  efficient,  or  they  detect  targets  (faults)  earlier  on  in  the 
search  process  (Schoonard,  et  al.,  1973;  Boynton,  1960).  Johnston  (1965)  provided  evidence  to  suggest 
that  subjects  who  obtain  high  peripheral  acuity  scores  exhibit  relatively  shorter  search  time.  There  is 
evidence  that  the  visual  lobe  size  is  amenable  to  training  (Gramopadhye,  Palanivel,  Knapp,  and  Drury, 
1991).  There  is  no  evidence  that  better  inspectors  have  shorter  fixation  times,  only  that  they  make  fewer 
fixations,  presumably  because  of  the  larger  visual  lobe  size. 

The  implication  for  aviation  inspectors  is  that  individual  differences  may  be  quite  large,  but  are  amenable 
to  training.  Other  evidence  (Gallwey,  1982;  Drury  and  Wang,  1986)  suggests  that  selection  tests  for  visual 
lobe  size  may  well  be  task-specific,  in  that  the  ability  to  search  for  defect  (  D  )  in  background  (  B  ) 

may  be  unrelated  to  the  ability  to  search  for  a  different  defect  i  D'  )  in  a  different  background  {  B'  ). 
As  Drury  and  Gramopadhye  (1990)  have  noted,  training  appears  to  be  a  more  powerful  intervention 
strategy  than  selection  for  inspection  tasks. 

5.3.4.2.2  Search  Strategy  Factors 

As  noted  earlier  in  this  section,  search  strategy  can  be  modeled  as  random  or  systematic,  with  humans 
believed  to  lie  in  between  these  two  extremes.  A  systematic  search  strategy  is  always  more  efficient  than 
a  random  strategy.  Scanning  strategy  is  dependent  on  an  inspector’s: 

1.  Familiarity  with  the  task  (experience). 

2.  Ability  to  obtain  and  utilize  feedforward  information  from  cues  regarding  defect  locations  and 
defect  types  uncertainty).  Gould  and  Cam  (1973)  and  Monk  (1977)  have  shown  that  in  tasks 
which  do  not  lend  themselves  readily  to  the  adaptation  of  systematic  search  strategy,  search  times 
increase  with  increased  fault  uncertainty. 

Search  strategy  in  visual  search  is  a  global  term  which  reflects  many  parameters  of  saccadic  movement. 
The  speed  with  which  search  is  performed  is  dependent  on  the  eye  movement  parameters,  such  as  tho.se 
listed  by  Megaw  and  Richardson,  1979;  fixation  times,  spatial  distribution  of  fixation,  interfixation 
distance,  duration  of  eye  movements,  and  sequential  indices.  Fixation  times  have  already  been  considered 
in  the  previous  section  on  visual  lobe  factors. 

Inspectors  do  not  have  uniform  coverage  of  the  area  in-spected  (Schoonard,  et  al..  1973).  with  the  central 
portions  given  more  attention  than  the  edges.  In  addition,  inspectors  may  not  always  choose  a  correct 
distance  between  successive  fixation  centers  (Gould  and  Schaffer,  1967;  Megaw  and  Richardson.  1979). 
The  scan  path  of  an  inspector  changes  with  experience  (Kundel  and  Fafollette,  1972;  Bhatnager,  1987) 
to  reflect  a  more  consistent  path,  more  even  coverage,  and  more  coverage  where  there  is  a  higher 
probability  of  a  fault  being  located. 
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The  studies  of  search  strategy  are  not  conclusive  on  how  to  take  practical  steps  to  improve  that  strategy, 
although  they  do  point  to  structuring  of  the  search  field  as  a  way  to  increase  the  likelihood  of  systematic 
search. 

With  a  structured  field,  the  current  fixation  point  will  serve  as  a  memory  aid  to  which  areas  have  already 
been  searched.  Suitable  structuring  devices  may  be  panel  lines,  physical  elements  of  complex  parts  (doors, 
landing  gear),  or  superimposed  temporary  structures,  such  as  inspectors’  markings  on  aircraft. 

Any  such  structuring  lines  should  be  made  clear  on  the  graphics  included  with  workcards,  and  in  any 
training  materials. 

There  are  likely  to  be  large  individual  differences  in  search  strategy,  differences  which  are  relatively  stable 
over  time.  The  issue  of  training  of  search  strategy  is  the  subject  of  one  of  the  experiments  presented  in 
the  training  section  (Section  5.3.5). 

5.3.4.2.3  SATO  and  Stopping  Policy  Factors 

Choice  of  operating  pxiint  on  the  SAOC  is  determined  by  the  perceived  utilities  of  speed  and  accuracy. 
The  only  error  pxissible  on  a  search  task  is  a  Miss,  so  that  high  accuracy  implies  locating  all  petential 
defects  in  the  structure.  Inspectors  are  highly  motivated  for  accuracy,  as  noted  earlier  (Shepherd,  et  al., 
1991),  so  that  one  would  expect  an  operating  peint  on  the  SAOC  representing  long  search  times,  with 
repeated  search  being  common.  In  practice,  inspectors  appear  to  stop  at  the  end  of  a  single  scan  of  the 
area,  only  repeating  a  fixation  if  some  indication  has  been  found.  It  appears  that  inspectors  recognize  the 
’’diminishing  returns"  aspect  of  search  performance,  and  are  confident  enough  in  their  abilities  that  a  single 
scan  at  the  appropriate  level  of  detail  is  seen  as  optimal.  Such  a  pelicy  certainly  reduces  the  memory  load 
and  petential  vigilance  effects  associated  with  multiple  scans.  However,  the  insp)ector  will  need  to  be 
"recalibrated"  at  pxiriodic  intervals  by  retraining  or  by  providing  test  sessions  to  ensure  that  the  speed  of 
insf)ection  chosen  is  appropriate  to  the  accuracy  demanded. 

5.3.4 J  Factors  Affecting  Decision  Making 

Decision  making  is  the  task  during  which  any  pxrtcnlial  defect  (indication)  located  by  the  search  task  is 
evaluated  to  determine  whether  it  should  be  repx)rted.  In  this  task  both  Typx;  1  errors  (False  Alarms)  and 
Typx;  2  errors  (Mis.ses)  can  occur.  These  have  their  own  tradeoff  relationship,  so  that  some  combined 
accuracy  measure  must  be  derived  before  any  tradeoff  between  spxjed  and  accuracy  can  be  considered. 

One  particular  model  of  the  human  as  a  rational  economic  maximizer  which  has  received  widespread 
support  in  inspection  is  Signal  Detection  Theory  (SDT).  Originally  propxised  by  Swets  and  various 
co-workers  (e.g.,  Swets,  1967)  as  a  model  for  how  humans  detect  signals  in  noise,  it  was  sub.sequcntly 
applied  successfully  to  inspxjction  (Wallack  and  Adam.s,  1969,  1970;  Sheehan  and  Drury,  1971;  Drury 
and  Addison,  1973). 

In  the  SDT,  the  inspxictor  is  as.sumed  to  be  making  a  choice  for  each  item  inspx'cted  of  whether  the  item 
contains  a  defect  (".signal")  or  dc^es  not  ("noise").  As  the  evidence  for  signal  or  noise  is  .somewhat 
equivocal,  there  is  assumed  to  be  an  "evidence  variable"  which  increases  when  a  signal  is  present  and 
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decreases  when  only  noise  is  present.  An  example  would  be  the  judgement  of  whether  a  dent  in  a 
stabilizer  leading  edge  should  be  reported.  Dents  can  range  from  almost  imperceptible  to  obviously 
reportable.  The  evidence  variable  (dent  visual  severity)  must  be  Judged  against  both  written  size  standards 
and  the  likely  effect  of  the  dent  on  flight  characteristics. 

SDT  shows  that  the  two  error  probabilities,  p  (miss)  and  p'  (false  alarm),  can  be  derived  from  a  model 
in  which  the  inspector  chooses  a  criterion  (X^)  to  report  on  the  presence  of  a  defect.  As  this  criterion 
varies  from  high  (defects  rarely  reported  as  present)  to  low  (defects  often  reported  as  present),  an 
Operating  Characteristic  Curve  is  traced  out.  This  curve  has  become  known  as  the  Receiver  Operating 
Characteristic  (ROC)  in  SDT  literature.  A  different  ROC  curve  is  traced  out  for  different  levels  of 

signal/noise  ratio,  known  as  discriminability  and  symbolized  hy  d'  . 

Wickens  (1984)  has  divided  tasks  into  those  which  are  resource  limited  and  those  which  are  data  limited. 
In  the  former  tasks,  as  the  operator  brings  more  resources  to  bear  on  a  problem  (e.g.  devotes  greater  lime 
to  it)  performance  improves.  In  a  data  limited  task,  the  quality  of  the  data  received  by  the  operator  is  the 
limiting  factor,  so  that  more  resources  yield  no  better  performance.  It  appears  that  SDT  tasks  are  only 
resource  limited  up  to  short  limes,  after  which  they  are  data  limited.  Because  aircraft  inspection  is 
typically  a  matter  of  minutes  and  hours  rather  than  seconds,  a  reasonable  assumption  is  that  its  decision 
making  aspects  are  data  limited.  Thus  there  is  unlikely  to  be  a  marked  SATO  for  decision  making  during 
the  inspection  task.  However,  the  grosser  aspects  of  decision  may  still  show  a  SATO.  For  example,  if 
the  inspector  is  unable  to  reach  a  decision,  the  supervisor  (or  other  senior  personnel)  may  be  called  in  to 
assist.  Here  the  inspector  is  attempting  to  improve  accuracy  at  the  cost  of  increased  time. 

From  the  SDT  model,  there  are  three  groups  of  factors  which  can  affect  the  overall  speed  and  accuracy: 

1.  Discriminability  or  sensitivity. 

2.  Choice  of  criterion. 

3.  Choice  of  SATO  operating  pxtint. 

5.3.43.1  Factors  Affecting  Sensitivity 

Most  factors  affecting  discriminability  or  sensitivity  are  physical,  and  can  be  characterized  as  the  perceived 
difference  between  the  observed  indication  and  a  standard.  Thus,  indications  obviously  well  above  or 
below  the  standard  will  have  high  d'  values.  Examples  would  be  large  areas  of  corrosion,  cracks 
noticeably  larger  than  those  allowed,  or  completely  missing  rivets.  None  would  require  difficult  (i.e.,  error 
prone)  decisions.  But  "pxirceived  difference"  implies  both  high  signal  and  low  noise  in  SDT  terminology. 
Low  noise  means  low  levels  of  visual  distraction  (i.e.,  competent  cleaning),  low  levels  of  fatigue  (i.e., 
frequent  task  breaks),  and  very  clear  standards  (i.e.,  well-defined  and  well -presented  Job  aids).  All  of 
these  can  be  improved  in  aircraft  inspection. 

Comparison  standards  at  the  work  place  have  been  shown  to  be  effective  in  improving  discriminability 
(Drury,  1990b).  It  should  be  possible  for  the  inspector  to  make  a  direct  side-by-side  compari.son  of  an 
indication  with  a  standard.  For  example,  the  critical  amount  of  corrosion  beyond  which  a  report  must  be 
made  should  be  indicated  by  a  life-sized  diagram  on  the  workcard.  Also,  if  different  corrosion  types  are 
present,  life-sized  photographs  help  in  positive  identification  (Harris  and  Chaney,  1969). 
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5.3.4J.2  Factors  Affecting  Criterion 

From  SDT,  the  two  factors  affecting  the  choice  of  criterion  are  the  relative  costs  of  errors  (misses  and 

false  alarms)  and  the  true  rate  of  defects  (p^.  From  these  factors,  the  optimum  criterion  can  be 
calculated,  but  this  is  rarely  the  exact  criterion  used  by  the  inspector.  In  laboratory  tasks,  and  in  non¬ 
aviation  inspection  tasks,  inspectors  choose  a  criterion  in  a  conservative  manner.  Thus,  if  the  criterion 
should  be  low  (i.e.,  they  should  be  very  willing  to  report  indications  as  defects),  inspectors  choose  a 
criterion  which  is  not  low  enough.  Similarly,  they  choose  a  criterion  which  is  not  high  enough  when  the 
criterion  should  be  high.  Because  of  this  conservatism  inspectors  may  not  react  quickly  enough  in 
changing  their  criterion  as  costs  and  probabilities  change.  Thus,  it  is  important  to  provide  accurate  and 
up-to-date  feedforward  information  on  the  probabilities  of  defects  in  different  areas  to  allow  the  inspector 
to  make  rapid  criterion  changes. 

There  are  also  known  criterion  shifts  with  both  changing  defect  rate  and  time  on  task.  TTiere  is  little  to 
be  done  about  increasing  the  defect  rate:  it  is  fixed  by  the  state  of  the  aircraft.  The  reduction  in  hit  rate 
at  very  low  defect  rates  may  well  set  a  limit  to  the  use  of  humans  as  detectors  of  rare  events. 
Paradoxically,  as  maintenance  improves  to  give  fewer  defects,  the  capability  of  the  inspector  to  detect  the 
few  remaining  defects  worsens.  There  is  clearly  a  need  for  more  research  into  human/machine  function 
allocation  to  alleviate  this  tow  defect  rate  problem.  Time  on  task,  the  vigilance  phenomenon,  only  causes 
a  reduced  detection  rate  due  to  criterion  shift  under  special  circumstances,  i.e.  uninterrupted  performance. 
This  may  not  be  a  problem  in  aircraft  inspection,  although  the  heavy  use  of  night  shift  inspection  where 
interruptions  are  less  frequent  and  the  human  less  vigilant,  requires  further  study. 

5.3.4J.3  Factors  Affecting  SATO 

The  influence  of  decision  time  on  sensitivity  (d^  was  seen  earlier,  where  it  was  suggested  that  it  may 
not  be  of  great  importance.  The  ability  of  the  inspector  to  integrate  signal  information  over  time  may  only 
extend  for  very  short  periods,  at  least  compared  to  the  time  spent  on  search.  However,  this  signal 
integration  is  not  the  only  temporal  aspect  of  decision  making.  When  an  indication  is  found  from  search, 
time  is  taken  not  so  much  in  obtaining  signal  input  as  in  locating  and  using  standards,  and  performing  the 
response.  Thus,  an  inspector  may  have  to  locate  the  relevant  standard  on  the  workcard  (which  is  a 
relatively  rapid  task)  or  in  a  manual  (a  longer  task),  or  even  through  interpretation  by  others  in 
management,  quality  control,  or  engineering  (a  much  longer  task).  The  response  requires  time  to  write, 
and  a  memory  load.  This  response  will  also  produce  more  work  for  the  maintenance  team,  and  hence 
potentially  delay  return  to  service.  All  of  these  represent  indirect  time  pressures  on  the  inspector. 

In  practice,  inspectors  do  not  appear  to  respond  to  such  time  pressures  as  much  as  may  be  expected.  Their 
training  and  management  reinforcement  is  biased  towards  accuracy  in  any  SATO.  However,  the  managers 
of  inspectors  do  feel  these  pressures,  and  also  feel  the  need  to  insulate  "their"  inspectors  from  the 
pressures. 
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5.3.4.4  A  General  Framework  for  Improving  Speed/Accuracy  Tradeoff 

In  this  section,  a  wide  variety  of  temporal  effects  on  inspection  have  been  noted.  In  addition  to  the  direct 
effect  of  time  pressures  (SATO),  effects  of  time-on-task  and  time-of-day  can  be  expected  where  vigilance 
or  fatigue  are  relevant  issues. 

The  main  focus,  however,  has  been  on  the  joint  performance  measures  of  time-per-item  and  inspection 
errors,  or  their  complements— speed  and  accuracy.  Models  have  been  presented  which  show  how  speed 
and  accuracy  are  Jointly  determined.  Access,  search,  and  decision  making  all  show  a  predictable 
speed/accuracy  tradeoff. 

If  the  objective  is  ultimately  to  bring  speed  and  accuracy  jointly  under  control,  then  the  same  concepts 
apply  to  all  three  key  tasks.  The  equations  deflning  the  speed/accuracy  operating  characteristics  have  been 
given  in  detail,  but  the  essence  of  all  is  the  same:  the  SAOC  defines  the  envelope  of  possible  performance, 
determined  by  the  physical  functioning  of  the  human  operator  within  a  physically-defined  system.  The 
choice  of  operating  pwint  on  the  SAOC  is  determined  by  the  perceived  costs  of  time  and  errors,  and  by 
the  perceived  probability  of  a  defect  being  present.  Thus,  there  are  two  control  modes,  hopefully  applied 
in  sequence: 

1.  Obtain  the  best  SAOC  envelope. 

2.  Obtain  the  best  operating  point  on  the  envelope. 

Clearly,  the  first  control  mode  gives  the  prospect  of  simultaneous  improvement  in  speed  and  accuracy, 
whereas  the  second  control  mode  only  substitutes  one  undesirable  consequence  (time)  for  another  (errors). 
The  analogy  with  inspection  instrumentation  (a  close  analogy  for  decision  making)  is  that  the  first  control 
mode  represents  increasing  the  signal-to-noise  ratio  of  the  instrument,  while  the  second  control  mode  is 
equivalent  to  choosing  an  optimum  threshold  setting. 

The  first  control  mode  can  be  represented  for  the  three  tasks  considered  as: 

Access:  Changing  the  controllability  of  the  vehicle  or  the  unaided  human  movement. 

Search:  Changing  the  visual  lobe  size,  area  to  be  searched,  and  fixation  time. 

Decision  Making:  Changing  the  sensitivity/discriminability  of  the  defect. 

Alt  of  these  three  parameters  {k,  t,  d')  will  take  effort  to  improve,  as  they  imply  a  change  in  either  the 
physical  system  or  the  human  training  to  deal  with  that  system.  The  benefit  from  these  changes,  however, 
is  seen  in  both  speed  and  accuracy,  and  will  be  obtained.  However,  the  specd/accuracy  tradeoff  is  set 
(within  broad  limits). 

In  contrast,  the  second  control  mode  implies  altering  the  human’s  perception  of  costs/payoffs  and 
probabilities  to  ensure  that  the  balance  the  inspector  chooses  between  speed  and  accuracy  is  the  one  which 
is  optimal.  For  all  of  the  models,  this  comes  down  to  the  costs  and  probabilities  of  errors  and  the  costs 
of  time.  Error  costs  come  from  peers  and  other  co-workers,  from  the  management,  and  ultimately  from 
society  and  its  institutions  (e.g.,  FAA)  Costs  of  time  come  from  perceived  urgency  of  job  completion. 
Examples  are  gate  pressures,  and  the  requirement  for  inspection  to  be  completed  early  so  that  repairs  can 
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be  scheduled.  If  there  are  conflicts  and  inconsistencies  between  these  costs  from  different  sources,  or  even 
their  perceived  costs,  then  confusion  and  inconsistency  will  result.  For  all  convex  SAOC  curves, 

averaging  of  two  different  operating  points  will  produce  an  apparent  operating  point  (C)  on  a  lower 
SAOC.  Inconsistency  in  the  second  control  model  can  thus  appear  as  a  worsening  in  the  first  control 
mode. 

Control  of  perceived  costs  is  largely  a  function  of  the  organization:  its  structure  and  its  information  flows. 
With  a  complex  system  such  as  aircraft  maintenance  and  inspection  (e.g.,  Taylor,  1990),  intervention  must 
follow  careful  technical  analysis  of  the  organization.  For  example,  the  more  separated  the  inspection 
subsystem  is  from  the  maintenance  subsystem,  the  fewer  the  direct  pressures  on  the  inspector.  However, 
the  price  of  this  independence  may  well  be  lack  of  coordination  and  technical  understanding  between  two 
of  the  major  groups  involved  in  maintaining  airworthiness.  Observations  made  during  this  project  have 
pointed  towards  a  lack  of  perceived  time  pressure  on  inspectors,  largely  due  to  their  managers’  function 
as  insulators.  No  quantitative  data  (e.g.,  from  surveys,  questionnaires,  or  ratings)  are  available  to 
substantiate  this  observation,  but  an  obvious  next  step  is  to  collect  such  data  in  a  formal  manner.  The 
outcome  of  such  a  data  collection  effort  would  be  a  baseline  of  how  (and  where)  inspectors  choose  their 
operating  point  on  the  SAOC.  The  options  available  for  changing  the  SAOC  and  the  operating  point  are 
still  those  given  in  this  section. 

5.3J  A  FRAMEWORK  FOR  TRAINING  FOR  VISUAL  INSPECTION 

In  parallel  with  development  of  training  sy.stems  for  diagnostic  tasks  (e.g.,  Johnson.  1990)  the 
predominance  of  visual  inspection  requires  studies  of  visual  inspection  training.  Earlier  reviews  of  training 
in  aircraft  inspection  (Drury  and  Gramopadhye.  1990;  Shepherd,  et  al..  1991)  have  shown  how  the 
component  tasks  of  inspection  are  amenable  to  training  interventions.  Literature  from  industrial  inspection 
training  was  reviewed  and  applied  to  aircraft  inspection. 

Training  is  aimed  at  reducing  both  search  errors  (all  misses)  and  decision  errors  (misses  and  false  alarms). 
From  a  review  of  the  various  training  interventions  available  (Gramopadhye.  1992),  it  becomes  apparent 
that  some  interventions  are  better  suited  to  some  component  tasks.  The  following  section  presents  part 
of  this  review  as  a  re.search  rationale  which  will  lead  to  specific  experimental  tests  of  training 
interventions.  The  review  in  Section  5J.5.1  covers  three  areas  which  are  critical  to  inspection 
performance:  search,  decision-making,  and  perception. 

5.3^.I  Results  of  Inspection  Training  Literature  Review 

5.3,5.1.I  Search 


As  noted  in  Section  5.3.4.2,  search  task  performance  is  a  function  of  visual  lobe  size  and  search  strategy. 
Visual  lobe  training  has  been  studied  by  Leachtenaver  (1978)  for  photo-interpreters,  who  found  that 
practice  on  a  search  task  increased  visual  lobe  size.  However,  practice  on  a  visual  lobe  measurement  task 
may  also  increase  lobe  size  and  transfer  this  increase  to  search  performance. 

Search  strategy  training  is  an  under-represented  area  in  the  literature.  From  the  literature  it  is  seen  that 
systematic  search  is  always  more  efficient  than  random  search,  .so  that  a  useful  assumption  is  that  the 
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searcher  is  always  trying  to  be  systematic  (Arani,  Drury  and  Karwan,  1984).  One  training  objective 
should  be  to  ensure  systematic  search,  i.e.  search  in  which  all  areas  are  fixated,  and  none  are  refixated 
during  a  single  scan.  The  major  difference  between  systematic  and  random  search  is  whether  or  not  an 
area  is  refixated.  The  only  logical  reason  for  an  inspector  to  refixate  an  area  before  a  total  scan  is 
completed  is  that  the  searcher  does  not  remember  whether  or  not  that  area  has  been  fixated  already. 
Hence,  it  is  seen  that  it  is  necessary  to  provide  a  memory-aid  to  the  inspector  to  indicate  the  points  of 
previous  fixations  to  avoid  refixations.  This  could  be  done  by  training  the  inspectors  to  use  feedback  from 
eye  movements,  either  continuously  (on-line),  or  in  a  discrete  manner  at  the  end  of  a  search  task. 

Feedback  from  eye  movements  can  be  provided  regarding  both  the  number  of  fixations  and  the 
interfixation  distance.  Literature  suggests  that  these  parameters  are  correlated  with  an  inspector’s 
efficiency  in  locating  possible  defects.  Providing  this  sort  of  feedback  would  be  expected  to  result  in  the 
inspector  developing  a  more  efficient  search  strategy. 

5.3,5.!. 2  Decision  Making 

Wickens  (1984)  states  that  training  for  decision  making  can  be  provided  in  the  following  ways: 

•  Make  the  decision  maker  aware  of  the  nature  of  limitations  and  biases.  Training  operators  to  consider 
alternative  hypotheses  might  reduce  the  likelihood  of  cognitive  tunnel  vision. 

•  Provide  comprehensive  and  immediate  feedback  so  that  the  operators  are  forced  to  attend  to  the  degree 
of  success  or  failure  of  their  rules. 

•  Capitalize  on  the  natural  efforts  of  humans  to  seek  causal  relationships  in  integrating  cues  when 
correlations  between  variables  are  known  beforehand.  Hence,  providing  information  to  the  operator 
so  as  to  emphasize  the  co-relational  structure  would  help  in  entertaining  particular  hypotheses. 

5.3.5.1.3  Perception 

When  the  separate  features  that  define  all  objects  within  a  category  may  be  variable,  objects  are  assigned 
to  different  perceptual  categories.  Thus,  the  operator  needs  to  develop  a  perceptual  schema,  a  form  of 
knowledge  or  mental  representation  that  people  use  to  assign  to  ill-defined  categories.  The  schema  is  a 
general  body  of  knowledge  about  the  characteristics  of  a  perceptual  category  that  docs  not  contain  a  strict 
listing  of  its  defining  features  (e.g.,  features  which  must  all  be  present  for  a  particular  instance  to  be 
termed  a  category).  Because  of  such  fuzzy  defining  characteristics,  the  schema  is  normally  acquired  as 
a  result  of  perceptual  experience  with  examples  rather  than  learning  a  simple  defining  set  of  rules. 

According  to  Posner  and  Keele  (1968,  1970)  the  development  of  a  schema  consists  of  two  components: 

•  a  general  representation  of  the  mean,  i.e.,  the  basic  form  from  which  all  the  forms  are  derived; 

•  an  abstract  representation  of  the  variability. 

Research  in  schema  formation  suggests  that  the  nature  of  mental  representation  which  people  u.se  to 
classify  stimuli  into  categories  is  not  a  strict  list  of  the  characteristics  of  the  prototype  but  that  the  mental 
representation  also  contains  information  concerning  the  variability  around  the  template.  This  is  suggested 
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by  Posner  and  Keele  (1968)  who  found  that  exposure  to  a  variety  of  instances  of  a  schema  induced  better 
performance  than  repeated  exposure  to  a  single  instance. 

Theories  proposed  by  Medin  and  Schaffer  (1978)  state  that  assignment  is  not  made  by  relating  each  new 
instance  to  a  central  prototype  but  rather  relating  it  to  the  exemplar  to  which  it  is  most  similar  and  then 
assigning  each  new  instance  to  the  residence  category  of  that  exemplar. 

Thus,  from  the  above  discussion,  it  is  seen  that  to  help  in  the  development  of  the  schema  the  training 
provided  should  be  of  variable  instances  of  the  category  rather  than  a  single  instance  of  a  prototypical 
member  or  rules  defining  the  features  which  would  classify  the  members  into  categories.  The  amount  of 
variability  provided  in  the  training  should  be  similar  to  that  existing  in  the  real  setting. 

5.3,5.2  Rationale  for  Research  on  Visual  Inspection  Training 

From  the  above  discussion,  training  for  visual  search  would  be  expected  to  result  in  reduced  search  errors 
(Type  2  errors)  and  reduced  search  time.  Similarly,  training  for  decision  making  and  perception  would 
be  expected  to  result  in  reduced  Type  1  and  Type  2  errors.  Although  training  can  be  used  to  improve 
visual  inspection  performance,  specific  training  schemes  are  not  associated  with  factors  that  determine 
improvement  in  visual  inspection  performance.  Hence,  ad  hoc  training  schemes  are  developed  that 
guarantee  improvements  for  a  particular  task  without  consideration  whether  such  a  training  scheme  could 
be  extended  to  a  similar  task  or  a  different  task,  or  whether  the  training  is  optimizing  the  use  of  instructor 
and  trainee  time.  Hence,  the  first  step  in  the  development  of  a  rational  training  scheme  is  to  identify  the 
factors  that  affect  visual  inspection  performance.  The  next  step  is  to  determine  which  of  the  functions  of 
the  inspection  task  are  trainable.  This  in  turn  will  establish  the  sensitivity  of  the  inspection  parameters 
to  training. 

For  any  training  scheme  to  be  effective  it  should  minimize  both  search  errors  and  decision  errors.  Thus, 
referring  to  the  earlier  proposed  model  of  visual  inspection,  it  is  observed  that  intervention  strategies  could 
be  developed  at  various  stages  of  the  inspection  process  which  could  be  hypothesized  to  change  the 
inspection  parameters,  resulting  in  improved  performance. 

The  following  factors  are  critical  to  the  search  process: 

•  ability  to  identify  salient  features  which  can  be  associated  with  a  particular  defect  (so  that  features 
can  be  searched  in  parallel  instead  of  requiring  foveal  attention); 

•  visual  lobe: 

•  eye  movement  scanning  strategy. 

In  order  to  improve  visual  inspection  performance,  it  is  necessary  to  develop  training  schemes  which 
predict  improvements  in  the  above  factors.  In  the  following  section  various  training  schemes  are  briefly 
described. 
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Visual  Lobe  Training 

The  visual  lobe  is  a  Very  important  determinant  of  search  performance.  Johnston  (1965)  states  that 
observers  with  a  larger  visual  lobe  require  fewer  fixations  than  observers  with  a  smaller  visual  lobe.  He 
concluded  that  a  large  visual  lobe  or  peripheral  acuity  may  account  for  superior  search  performance.  We 
still  need  to  know  how  a  large  visual  lobe  can  affect  search  performance  and  how  people  can  be  trained 
so  as  to  increase  the  size  of  the  visual  lobe.  If  the  above  questions  are  answered,  this  would  then  result 
in  a  strategy  for  improving  the  visual  lobe.  The  more  general  question  which  arises  is:  how  does  lobe 
size  training  generalize  aaoss  tasks  (e.g.,  targets  and  backgrounds).  We  are  interested  in  understanding 
whether  the  visual  lobe  training  on  a  given  target  type  would  result  in  an  improved  search  performance 
for  a  different  target  type  and  the  sensitivity  of  the  search  parameter  to  this  type  of  training.  Thus,  it  is 
essential  to  identify  whether  such  a  cross-over  effect  exists.  If  it  does,  then  it  is  sufficient  to  train  the 
person  on  one  target  type.  If  not,  then  it  is  essential  to  identify  various  target  subsets,  say  Tj,  Tj,  within 
which  cross-over  does  occur.  The  people  could  be  provided  visual  lobe  training  on  a  single  target 
belonging  to  each  target  subset. 

5.3,5,2.2  Feedback  Training 

A  person  needs  rapid  and  accurate  feedback  in  order  to  correctly  classify  a  defect,  or  to  know  the 
effectiveness  of  a  search  strategy.  Every  training  program  should  begin  with  frequent  feedback  and 
gradually  delay  this  until  a  level  of  proficiency  has  been  reached.  Additional  feedback  beyond  the  end 
of  the  training  program  will  help  to  keep  the  inspector  calibrated  (Drury  and  Kleiner,  1990).  The 
following  feedback  could  be  provided: 

•  Feedback  regarding  the  correctness  of  classifying  defective  items  into  categories. 

•  Feedback  of  search  strategy  from  monitoring  eye  movements. 

•  Feedback  of  fixation  times  from  the  eye  movement  search. 

The  first  is  known  to  be  essential  to  learning  in  perceptual  tasks  (Annett,  1966).  It  provides  the  novice 
information  regarding  the  critical  difference  between  a  defective  item  and  perfect  item,  thus  helping  to 
develop  a  mental  template  which  has  the  internal  characteristics  of  the  defective  item.  We  are,  however, 
still  unsure  as  to  what  has  improved.  For  example,  has  learning  resulted  in  a  new  internal  conceptual 
model  of  the  task  (i.e.,  is  the  inspector  using  only  certain  dimensions  of  the  fault  to  classify  it)? 

It  has  been  shown  that  an  important  difference  between  the  best  and  the  poorest  search  performance  is  the 
length  of  the  sweeps  between  eye  fixations  during  a  search  task  (Boynton,  Elworth,  and  Palmer,  1958). 
Thus,  there  exists  a  difference  between  how  a  novice  and  an  expert  move  their  eyes  across  the  visual  field. 
Gould  (1973),  in  a  visual  inspection  study  of  circuit  chips,  found  that  most  of  the  eye  fixations  occur 
within  a  definite  boundary,  which  is  the  area  most  likely  to  contain  the  targets.  It  is  demonstrated  that 
eye  movements  in  a  visual  search  scenario  occur  based  on  knowledge  of  the  location  of  faults  and  on  the 
probability  of  them  occurring.  The  question  that  needs  answering  is:  does  feedback  information  regarding 
the  eye  movements  help  improve  the  scanning  strategy?  Here  we  hypothesize  that  providing  such 
feedback  information  would  aid  the  inspectors  by  allowing  them  to  identify  areas  not  covered  or  areas 
where  one  spends  excessive  time,  and  helping  them  develop  a  strategy  to  cover  the  entire  area  more 
effectively. 
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5.33J.3  Feedforward  Training 

When  a  novice  inspector  has  no  knowledge  of  the  type  of  faults,  probability  of  faults,  and  occurrence  of 
faults,  visual  search  would  be  expected  to  be  inefficient.  Providing  feedforward  information  should  result 
in  an  improved' search  strategy  because  the  uncertainty  is  reduced  by  the  inspector  knowing  both  where 
to  look  and  what  to  look  for.  Perhaps  the  inspector  could  use  the  information  to  achieve  a  more 
systematic  search  strategy,  guided  by  the  knowledge  of  the  fault  characteristics.  The  inspector  could  use 
feedforward  information  in  the  following  ways:  1)  to  ignore  the  information  completely,  2)  to  selectively 
incorporate  some  of  the  information,  or  3)  to  incorporate  this  information  only  at  later  stages  of  inspection, 
that  is,  only  after  gaining  some  verification.  Kleiner  (1983)  suggests  that  experienced  inspectors  make 
use  of  feedforward  information  that  complements  their  sensitivity  to  the  fault.  If  the  fault  is  one  that  is 
not  easily  detected,  then  the  inspector  relies  heavily  on  the  information  provided.  According  to  McKeman 
(1989),  insfjection  tasks  that  will  most  likely  benefit  from  the  addition  to  prior  information  include  those 
in  which  the  value  of  the  fault  is  greater  than  the  value  of  inspection  time,  those  in  which  the  fault  is 
particularly  difficult  to  detect,  and  those  in  which  the  product  may  contain  rare,  detrimental,  and  easily 
overlooked,  faults. 

5.3.5.2.4  Attribute  Training 

Consider  an  item  A.  Let  the  item  be  faulty  on  attributes  Al,  A2,  A3  and  A4.  TTie  inspector  could  be 
trained  on  each  of  the  above  attributes.  Such  training  would  allow  the  inspector  to  set  a  response  criterion 
for  each  attribute.  The  training  should  be  generalizable  in  the  sense  that  the  inspector  should  be  able  to 
classify  the  items  as  defective  if  the  items  are  faulty  on  one  or  more  of  the  attributes.  The  inspector  could 
be  trained  on  which  attributes  to  match  first  based  on  the  probability  of  the  item  being  faulty  on  the 
attributes  and  the  ease  with  which  the  matching  occurs.  Experience  and  training  of  the  inspectors 
determine  how  defect  attributes  are  arranged  (Goldberg  and  Gibson,  1986). 

A  similar  training  scheme  has  been  proposed  by  Salvendy  and  Seymour  (1973)  for  developing  industrial 
skills.  Here,  separate  parts  of  the  job  are  taught  to  criterion,  and  then  successively  larger  sequences  of 
the  Job  are  integrated.  Czaja  and  Drury  (1981)  and  Kleiner  (1983)  used  such  progressive  part  training 
very  effectively  in  inspection. 

5.3.5.2.5  Schema  Training 

It  is  essential  that  the  subject  develop  a  valid  mental  template  (internal  representation)  schema  of  the  fault. 
The  key  to  the  development  of  a  schema  is  that  it  should  provide  for  successful  extrapolation  to  novel 
situations  which  are  still  recognizable  instances  of  the  schema. 

We  need  to  know  how  schemas  are  developed,  whether  inspectors  can  be  trained  to  develop  schemas,  and 
what  sort  of  training  (rule  based  or  knowledge  based)  should  be  provided  to  the  inspectors  for  effective 
development  of  such  schemas. 

The  effects  of  two  methods  of  training  need  to  be  evaluated  in  schema  development:  "active  training"  and 
"passive  training".  In  active  training,  the  in.spector  is  prc.sented  with  various  iastances  of  the  fault  and  no¬ 
fault,  and  has  to  classify  them  as  defecti ve/non-defective.  Feedback  is  provided  regarding  itie  correctness 
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of  classification.  In  contrast,  passive  training  is  where  the  inspector  is  merely  presented  with  various 
instances  of  the  faults  without  requiring  an  active  response. 

5.3^J  Testing  the  Visual  Inspection  Training  Framework 

In  order  to  test  whether  the  above  predictions  of  training  intervention/task  component  match  are  correct, 
a  sequence  of  five  experiments  are  to  be  undertaken  as  follows.  All  use  the  visual  inspection  simulator 
described  in  Section  5.3.1.  Brief  synopses  of  each  experiment  are  presented,  with  more  detail  given  for 
Experiment  5,  which  has  been  completed. 

Experiment!:  Feedback  Training.  This  compares  a  control  group  and  three  feedback  groups,  using  on¬ 
line  and  off-line  feedback  of  both  cognitive  factors  and  performance  factors  (c.f.  Section  5.3.3). 

Experiment  2:  Feedforward  Training.  Again,  a  control  condition  is  used  as  a  baseline  against  which  to 
compare  rule-based  feedforward,  knowledge-based  feedforward,  and  combined  feedforward. 

Experiment  3:  Attribute  Training.  Training  for  decision  making  using  attributes  training,  i.e.,  providing 
the  trainee  with  several  levels  of  severity  and  complexity,  is  compared  to  a  control  condition  where 
narrative  descriptions  are  provided  for  the  fault  attributes. 

Experiment  4:  Schema  Training.  Schema  development  will  be  encouraged  by  exposing  trainees  to  a  wide 
variety  of  schema  instances  (corrosion  levels  and  patterns)  in  both  acUve  and  passive  schemes. 

Experiment  5:  Visual  Lobe  Training.  This  experiment  tests  for  the  possible  cross-over  effects  on  the  size 
of  visual  lobe  measured  for  different  fault  types. 

The  objectives  of  this  experiment  were  to  determine  the  relationship  between  visual  lobe  and  search 
performance,  relate  changes  in  lobe  size  to  search  performance,  and  evaluate  the  effectiveness  of  lobe 
training.  In  particular,  the  experiment  measured  whether  crossover  effects  exist  in  visual  lobe  training. 
It  used  two  types  of  rivet  fault  (cracks  and  loose  rivets)  and  two  types  of  area  fault  (corrosion  and  dents) 
to  determine  whether  visual  lobe  training  on  one  fault  would  generalize  to  other  faults  of  the  same  or 
different  classes. 

5.3.5J.1  Method 

Twenty-four  subjects  were  used  for  this  study  and  were  randomly  assigned  to  four  different  groups,  Gl, 
G2,  G3  and  G4.  Subjects  were  tested  for  20/20  vision  and  color  blindness.  All  the  subjects  were 
administered  the  EFT  (Embedded  Figure  Test)  and  MFFT  (Matching  Familiar  Figure  Test),  which  have 
been  shown  to  correlate  with  different  a.spects  of  industrial  inspection  performance. 

Group  Gl;  Subjects  assigned  to  this  group  initially  performed  the  vi.sual  seaich  task  on  the  above  four 
fault  types  (randomly  ordered)  followed  by  visual  lobe  training  on  rivet  cracks.  The  visual  lobe  training 
coasisted  of  performing  the  visual  lobe  task  five  times.  The  training  session  was  followed  by  a  .search 
task  on  the  four  fault  types. 
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Group  G2:  Subjects  assigned  to  this  group  also  initially  performed  visual  search  tasks  on  all  four  targets 
(ordered  randomly).  They  followed  this  by  visual  lobe  training  on  one  area  fault  and  dent.  The  visual 
lobe  training  consisted  of  performing  the  lobe  task  five  limes.  The  training  session  was  followed  by  a 
search  task  on  all  four  fault  types. 

Group  G3:  Subjects  assigned  to  this  group  performed  the  visual  search  task  in  a  similar  manner  to 
subjects  in  Groups  G1  and  G2.  However,  this  was  followed  by  visual  lobe  training  on  a  neutral  target, 
a  computer-generated  character.  This  training  session  was  followed  by  a  similar  visual  search  task. 

Group  G4:  Subjects  assigned  to  this  group  performed  similar  visual  search  tasks.  However,  they  did  not 
undergo  any  visual  lobe  training.  Subjects  in  Group  4  performed  a  computer  task  for  a  duration  equal  to 
the  time  required  for  the  completion  of  the  visual  lobe  training  session  in  Groups  Gl.  G2,  and  G3.  This 
was  followed  by  a  visual  search  task. 

5.3.5J.2  Tasks 

Visual  Search  Task.  The  visual  search  task  was  the  simulated  airframe  visual  inspxiction  task  described 
in  Section  53.1.  Subjects  had  to  search  for  a  single  fault  type  in  a  given  area.  Visual  search  performance 
of  the  subjects  was  evaluated  on  four  faults  which  were  classified  into  two  types: 

1.  Area  Faults  -  1)  corrosion,  and  2)  dent 

2.  Rivet  Faults  -  1)  rivet  crack,  and  2)  loose  rivets  (indicated  by  streaks  of  dirt  on  the  rivet  edge). 

The  task  was  unpaced.  During  each  of  the  four  visual  search  tasks,  the  subjects  had  to  search  for  one  of 
the  predefined  faults.  Subjects  were  instructed  to  work  as  rapidly  as  possible  consistent  with  accuracy. 
Subjects  verified  their  response  by  clicking  on  the  fault  with  the  mouse  button.  Once  a  fault  was  located 
in  a  given  area  subjects  inspected  the  next  area. 

Visual  Lobe  Task.  The  purpose  of  the  lobe  task  was  to  determine  the  size  of  the  visual  lobe;  i.e.,  how 
far  into  the  periphery  a  subject  could  see  in  a  single  fixation.  The  basic  procedure  consisted  of 
determining  at  what  distance  from  the  central  fixation  point  the  target  was  completely  seen  by  the  subject 
in  a  single  fixation  of  the  fault  screen.  The  exposure  duration  was  kept  sufficiently  short  (0.33  s)  to  allow 
the  subject  a  single  fixation  only.  Subjects  had  to  identify  a  single  fault  (a  rivet  fault  in  group  Gl,  an  area 
fault  in  group  G2  and  a  neutral  fault  in  group  G3).  The  fault  would  appear  on  the  horizontal  center  line 
of  the  target  screen,  at  six  equally  spaced  predetermined  locations  on  the  horizontal  center  line,  three 
positions  on  either  side  of  the  central  fixation  point.  No  prior  information  concerning  the  position  of  the 
target  was  provided  to  the  subjects.  The  subjects  identified  the  position  of  the  target,  either  to  the  left  t)r 
to  the  right  of  the  origin  and  accordingly  pressed  the  key  "Q"  and  "P"  to  register  their  response.  Subjects 
were  requested  to  avoid  guessing  and  register  responses  only  if  they  were  sure  as  to  the  position  of  the 
targets.  The  fault  screen  alternated  with  a  fixation  .screen,  consisting  of  crosswires  at  the  central  fixation 
point  exposed  for  a  period  of  2  seconds.  The  purpose  of  the  fixation  screen  was  to  help  the  subjects  fixate 
in  the  center  of  the  screen  after  each  viewing  of  the  target  screen. 
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5.3^3.3  Hypotheses  Tested 

1.  Visual  lobe  training  on  one  rivet  fault  (rivet  crack)  will  result  in  improved  visual  search 
performance  in  detecting  rivet  faults  (rivet  cracks  and  loose  rivets). 

2.  Visual  lobe  training  on  one  area  fault  (dent)  will  result  in  improved  search  performance  in 
detecting  area  faults  (dents  and  corrosion). 

3.  Search  performance  on  a  fault  will  be  superior  in  the  case  of  subjects  who  underwent  visual  lobe 
training  on  the  particular  fault  than  for  subjects  who  under  went  training  on  a  neuU'al  target,  or 
subjects  who  did  not  undergo  any  visual  lobe  training. 

Hypotheses  1  and  2  tested  for  crossover  effects  of  visual  lobe  training  and  hypothesis  3  tested  for  the 
effectiveness  of  visual  lobe  training  in  improving  visual  search  performance. 

5.3^.3.4  Experimental  Design 

The  design  was  a  4  groups  x  2  trials  factional  design  with  six  subjects  nested  within  each  group.  The 
following  performance  measures  were  collected: 

1.  Number  of  correct  responses  for  each  of  the  six  fault  positions  in  the  visual  lobe  task. 

2.  Time  to  detect  a  fault  in  each  screen  for  the  visual  search  task. 

5.3^J.5  Results 

To  determine  whether  the  visual  lobe  increased  in  size  during  the  training,  an  Analysis  of  Variance 
(ANOVA)  was  conducted  for  the  lobe  size  for  the  three  groups  (1,2,  and  3)  receiving  lobe  training.  Over 
the  five  training  trials,  significant  effects  of  group  (F  (2,15)  =  1 1.05,  P  <  ().(X)1 1),  training  trial  (F  (4,60) 
=  13.46,  P  <  0.0000)  and  their  interaction  (F  (8,60)  =  1.75,  P  <  0.1046)  were  found.  To  test  whether  the 
visual  lobe  training  transferred  to  the  visual  search  task,  ANOVAs  were  performed  on  the  mean  search 
times  for  each  fault  type.  These  analyzes  are  summarized  in  Table  5.22,  showing  no  main  effects  of 
groups,  but  highly  significant  group  X  trial  interaction.  Figure  5.11  shows  these  group  X  trial 
interactions,  where  it  can  be  seen  that  the  two  faults  trained  in  the  visual  lobe  training  had  the  largest 
improvement.  For  the  faults  rtot  trained  by  visual  lobe  training,  the  improvement  was  greater  where  there 
was  more  similarity  to  the  visual  lobe  fault.  Neutral  training  had  a  smaller  amount  of  transfer,  while  no 
training,  i.e.,  spending  equivalent  time  on  other  computer  ta.sks,  had  no  beneficial  effect. 


Search  Time 

Analyzed 

Group  (G) 

Trial  (T) 

Group  X  Trial 

Loose  Rivet 

P  >  0.25 

P  <  0.005 

P  <  0.05 

Rivet  Crack 

P  >  0.25 

P  <  0.005 

P  <  0.10 

Dent 

P  >  0.25 

P  <  0.01 

P  <  0.05 

Corrosion 

P  >  0.15 

P  <  0.05 

P  <  0.05 

Overall 

P  >  0.25 

P  <  0.0001 

P  <  0.005 

Table  5.22  Summary  of  Analy.ses  of  Variance  of  Mean  Search  Times 
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Figure  5.11  Search  Performance  Before  and  After  Visual  Lobe  Training 
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Similar  results  can  also  be  seen  when  the  changes  in  visual  lobe  size  during  training  arc  related  to  the 
changes  in  search  time  after  training.  Table  5.23  relates  the  dependence  of  search  time  for  each  fault  type 
to  the  increases  in  lobe  size,  using  the  coefficient  of  determination  (r^)  as  the  measure  of  dependence. 


Percent  Increase  in 

Percent  Decrease  in  Search  Time  For; 

Visual  Lobe  Size 

For 

Loose  Rivet 

Rivet  Crack 

Corrosion 

Dent 

Group  1  (Loose  Rivet) 

0.75 

0.36 

0.01 

0.21 

Group  2  (Dent) 

0.09 

0.00 

0.68 

0.85 

Group  3  (Neutral) 

0.16 

0.05 

0.74 

0.00 

Table  5.23  Dependence  (r“)  of  Percent  Changes  in  Search  Time  on  Percent  Changes  in  Visual  Lobe 
Size  for  Each  Group 


There  was  a  direct  transfer  from  the  fault  used  in  visual  lobe  training  to  that  fault  in  visual  search,  with 
a  smaller  transfer  to  the  other  fault  in  the  same  group  (rivet  or  area).  The  neutral  fault  visual  lobe  training 
transferred  only  to  one  area  fault. 

5.3.5J.6  Di.scussion  and  Conclusions 

Providing  training,  even  just  repeated  practice,  in  rapidly  detecting  a  fault  in  peripheral  vision,  does  indeed 
increase  the  size  of  the  area  in  which  that  fault  can  be  detected  in  a  single  glimpse,  i.e.,  the  visual  lobe. 
This  increased  visual  lobe  is  not  merely  a  result  of  increased  familiarity  with  the  experimental  visual  lobe 
task,  as  it  transfers  to  a  more  realistic  inspection  task,  visual  .search.  Thus,  even  such  a  basic  aspect  of 
inspection  performance  as  the  visual  lobe  can  be  improved  through  haining.  For  each  fault  type  tliere 
was  a  increase  in  lobe  size  over  Ju.sl  five  practice  trials.  This  transferred  to  the  search  task  with 

percentage  changes  in  overall  visual  search  time  of; 


Group  1  (Loose  Rivet) 

.30'^ 

Group  2  (Dent) 

327r 

Group  3  (Neutral) 

187r 

Group  4  (No  Training) 

-49r 

Tliere  is  a  close  correspondence  between  the  training  on  actual  faults  (Groups  1  and  2)  and  improvement 
in  search  limes,  and  even  .some  improvement  for  training  on  a  neutral  fault,  i.e.,  one  which  did  not  appear 
in  any  .search  tasks.  No  training,  as  expected,  produced  no  effect. 

From  Section  5J.4,  it  was  seen  that  visual  .search  follows  a  speed/accuracy  tradeoff  curve,  so  that  what 
has  been  measured  here  as  search  times,  can  also  be  interpreted  as  search  accuracy  in  a  given,  fixed  time. 
Tlius,  this  experiment  has  demonstrated  the  value  of  training  in  increasing  the  inspector’s  ability  to  receive 
and  interpret  peripheral  visual  information.  Tlie  implication  is  that  tasks  similar  to  the  visual  lobe  task 
given  here  need  to  be  derived  and  u.sed  with  iaspectors.  Tlie  benefits  of  a  simple,  simulator-ba.sed  study 
in  rapidly  determining  the  feasibility  of  new  training  techniques  has  also  been  demonstrated.  A  study 
based  on  actual  faults  on  a  real  aircraft  structure  would  have  been  impo.ssible  as  single  glimpses  cannot 


1.34 


Human  Reliability  in  Aircraft  Inspection 


be  repeated  without  the  inspector  learning  the  true  identity  of  each  fault.  A  study  using  hardware  to 
simulate  the  faults  would  be  extremely  cumbersome,  with  hundreds  of  fuselage  samples  identical  apart 
from  fault  location  being  required. 

5.3.6  INTERNATIONAL  COMPARISONS  IN  AIRCRAFT  INSPECTION 

As  noted  in  Section  5.1,  a  joint  study  of  inspection  practices  in  the  U.K.  and  U.S.A.  was  undertaken  as 
part  of  a  Memorandum  of  Agreement  between  the  CAA  and  FA  A.  The  Lock  and  Strutt  report  (1985)  was 
in  fact  prepared  earlier  in  that  decade,  so  that  the  CAA,  for  whom  the  report  was  produced,  initiated  an 
update  by  M.  W.  B.  Lock  in  1990.  As  the  techniques  of  observation  were  similar  to  those  used  by  the 
FAA/AAM  team,  a  Joint  venture  was  created  to  allow  direct  comparison  of  U.S.A.  and  U.K.  practices. 
Both  C.  G.  Drury  and  M.  W.  B.  Lock  were  participants,  and  have  issued  a  joint  report  (Drury  and  Lock, 
1992),  so  that  only  a  briefer  summary  is  presented  here. 

The  aircraft  to  be  maintained  are  designed  and  sold  for  world-wide  markets,  so  that  much  of  the  inspection 
and  maintenance  is  pre-determined  by  the  manufacturers.  However,  the  various  regulatory  authorities 
around  the  world  (e.g.,  FAA,  CAA,  JAA)  have  different  requirements.  In  addition,  the  way  in  which  an 
airline  chooses  to  meet  these  requirements  leaves  some  latitude  for  local  and  cultural  variations. 

Although  many  points  of  difference  were  noted,  perhaps  the  most  obvious  is  in  the  way  in  which  the 
iaspection/maintenance  job  is  scheduled  and  conaolled.  In  the  U.K.,  the  management  structures  of 
maintenance  and  inspection  are  u,sually  closely  intermeshed.  In  the  past  it  was  frequently  the  case  that 
the  engineering  manager  and  the  quality  control  chief  were  the  same  person.  Although  this  not  the  case 
in  large  Uansport  aircraft,  it  can  still  be  the  case  in  smaller  commuter  airlines.  Work  arising  from  an 
inspection  can  be  allocated  by  the  inspector,  who  is  often  also  a  supervisor,  or  by  a  senior  person  who  has 
responsibility  for  both  inspection  and  maintenance.  The  inspector  is  frequently  consulted  during  the  defect 
rectification,  in  some  cases  is  the  actual  supervisor  of  that  work,  and  will  usually  be  the  person  to  buy 
back  the  repair. 

In  the  U.S.A.  the  management  structures  of  maintenance  and  inspection  are  separated  up  to  a  level  well 
beyond  the  hangar  floor.  A  wide  variation  of  management  authority  was  found  whereby  either 
maintenance,  inspection,  or  even  planning,  could  dominate  (Taylor.  1990).  In  a  few  companies  visited 
there  was  provision  for  coordination  between  maintenance  and  inspection  by  an  engineer  whose  job  was 
to  ensure  some  cross  talk.  The  engineer  served  as  .shift  change  coordinator.  Typically  though,  work 
arising  from  an  inspection  is  allocated  by  a  maintenance  supervisor  so  that  the  inspector  who  raised  the 
defect  has  no  responsibility  for  defect  rectification  and  may  not  be  Uie  inspector  who  does  the  buy-back 
inspection. 

The  separation  of  the  two  management  sU'uctures  in  the  U..S.A.  is  dictated  largely  by  the  existing  Federal 
Airworthiness  Regulations,  driven  by  a  deeply-felt  need  for  checks  and  balances  as  an  error  reduction 
mechanism.  At  the  hangar  flixir  level  the  general  view  is  that  repair  and  maintenance  would  suffer  if  the 
repairer  knew  that  certain  inspectors  were  ‘buying  back’  the  work,  as  .some  are  known  to  be  less  stringent 
than  others.  The  general  view  in  the  U.K.  was  that  the  .system  of  having  the  same  in.spector  respon.sible 
throughout  for  any  particular  defect  and  its  rectification  was  preferable  as  the  repair  could  be  monitored 
at  appropriate  .stages,  en.suring  that  the  job  had  been  performed  correctly. 
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Both  systems  lead  to  different  requirements  for  training  in  managerial  skills.  Despite  the  greater  direct 
management  responsibilities  of  inspectors  in  the  U.K.,  little  formal  training  in  managerial  skills  was 
evident. 

A  number  of  visits  were  undertaken  by  each  participant  in  each  country,  either  separately  or  together. 
There  was  no  attempt  at  comprehensive  sampling;  rather  the  knowledge  of  each  participant  was  used  to 
select  sites  which  would  be  illustrative  of  various  features.  For  example,  in  the  UK,  visits  were  made  to 
specialist  third-party  NDT  companies  which  serviced  civil  aviation  because  they  represent  a  major  source 
of  NDT  expertise  utilized  by  some  airlines. 

At  each  site,  the  visit  was  divided  into  two  sections,  although  these  often  overlapped  in  coverage; 

•  Systems  Overview.  First  the  management  of  the  maintenance  of  the  site  was  probed  in  management 
interviews.  The  structure  of  the  maintenance  and  inspection  organi/,ation(s)  was  elicited  during 
discussions  with  managers,  shift  supervisors,  foremen,  and  often  with  staff  who  were  outside  the  line 
management  structure.  These  could  include  training  personnel,  archive  keepers,  work  card  preparers, 
planners,  and  so  on  depending  upon  the  initial  discussions  with  management.  The  aim  was  to  be  able 
to  write  a  short  description  of  how  the  system  should  operate,  and  the  management  philosophy  behind 
this  system  structure  and  functioning. 

•  Hangar-Floor  Operations.  Detailed  observations  of  the  practice  of  inspection,  and  its  organizational 
constraints,  were  made  by  following  an  inspector  for  all  or  part  of  a  shift.  As  the  inspector  progressed 
through  a  Job,  questions  were  asked  concerning  the  inspection  itself  and  ancillary  operations,  such  as 
spares  availability  from  stores,  or  time  availability  for  training.  Thus  a  reasonably  complete  task 
description  and  analysis  could  be  written  on  the  inspection  task  itself,  while  obtaining  information  on 
the  wider  context  of  the  inspector’s  Job.  Tliis  technique  also  allowed  the  collection  of  anecdotal 
recollections  of  previous  Jobs,  and  other  events  from  the  past.  While  lhe.se  had  an  obviously  lower 
evidence  value  than  direct  observation  of  task  performance,  they  did  provide  a  valuable  adjunct  to  the 
data  collection  process. 

Sites  visited  included  major  air  carriers,  regional  or  second-level  airlines,  repair  stations,  and  NDT 
companies.  In  addition  visits  were  made  to  FAA  and  CAA  personnel  and  to  a  Royal  Air  Force  base 
where  maintenance  and  iaspection  procedures  are  written. 

5.4  CONCLUSIONS 

As  the  FAA/AAM  program  on  human  factors  moves  from  its  second  to  third  phases,  work  has  progressed 
from  observation  to  demonstrations  of  concepts  for  doing  maintenance  and  inspection.  Tlie  original 
approach,  developed  in  Phase  I  and  reported  in  Shepherd,  et  al.,  (1991)  was  to  have  human  factors 
engineers  .study  aircraft  inspection  and  maintenance  ,so  as  to  determine  a  strategy.  Hnough  depth  and 
breadth  of  study  was  maintained  to  be  able  to  find  critical  intersections  between  human  factors  knowledge 
and  techniques  on  one  hand,  and  field  problems  r)f  inspection  and  maintenance  on  the  other.  Tliis  involved 
both  top-down  analysis,  taking  a  sy.stems  view,  and  bottom-up  analysis,  performing  detailed  task  analy,ses 
of  inspector’s  Jobs. 
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Phase  II  has  rather  closely  followed  the  recommendations  made  in  Phase  I.  Observation  of  field  activities 
has  been  scaled  down  and  re-focussed  onto  very  specific  areas.  These  have  evolved  into  the  on-going 
sequence  of  demonstration  projects.  While  results  from  the  first  two  such  projects  are  not  scheduled  to 
be  available  until  the  summer  of  1992,  the  concept  appears  to  be  working  well.  Airline  personnel  at  all 
levels  recognize  that  improvements  are  possible,  and  thus,  are  being  most  cooperative  with  the  human 
factors  team. 

As  Phase  III  approaches,  more  of  the  projects  listed  in  the  Phase  I  report  will  be  performed,  as  well  as 
new  ones  added.  For  example,  the  whole  field  of  inspection  and  maintenance  scheduling  could  benefit 
from  human  factors  research  into  combined  human/automated  scheduling  systems  (e.g.,  Sanderson,  1989). 
When  projects  are  completed,  a  dissemination  of  results  and  lessons  learned  will  be  needed,  presumably 
by  presentations  and  published  papers.  Both  the  FAA  and  the  airline  maintenance  organizations  need  to 
consider  the  best  ways  for  rapid  dissemination  and  application  of  demonstration  project  results. 

The  detailed  application  of  human  factors  knowledge  (often  models)  to  specific  problems  (Sections  5.3.1- 
5.3.5)  has  yielded  insights  for  the  experimental  program  and  the  demonstration  projects.  Feedback  is  now 
required  from  the  industry  on  whether  finds  this  work  adds  to  its  operational  understanding.  The 
experimental  program  is  just  starting,  following  hardware  procurement  and  software  development.  As  this 
progresses,  the  same  simulations  should  be  available  for  specific  experiments  supported  by  industry,  as 
well  as  for  the  on-going  programs  presented  here. 

The  long-term  aim  of  the  whole  project  is  to  provide  phased  solutions  of  practical  u.se  to  industry  to 
improve  the  already  high  performance  of  aircraft  inspection  and  maintenance. 
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Chapter  Six 

A  Human  Factors  Guide  For  Aviation  Maintenance 
6.0  INTRODUCTION 

The  U.S.  air  carrier  industry  and  the  Federal  Aviation  Administration  are  dedicated  to  the  highest  level 
of  safety  in  commercial  aviation.  To  achieve  this  goal,  they  must  rely  on  effective  and  efficient 
maintenance  operations.  Proper  maintenance  support  is  indispensable  to  safety,  to  aircraft  availability,  and 
to  airline  profitability.  The  safety  requirements  dictate  that  maintenance  be  effectively  error-free.  Aircraft 
inspectors  and  aircraft  mechanics  must  work  in  an  environment  and  use  procedures  and  equipment  all 
carefully  structured  to  work  well  and  to  minimize  any  potential  for  error.  The  design  of  procedures  and 
equipment  must  ensure  that  errors  are  not  built  into  the  system. 

The  maintenance  effort  to  ensure  continuing  airworthiness  of  the  air  carrier  fleet  is  demanding  and  costly. 
The  maintenance  industry  continues  to  grow  in  parallel  with  that  of  airline  operations.  Table  6.1  shows 
that,  in  1991,  about  59,000  mechanics  were  employed  in  this  industry,  with  maintenance  expenses  of 
approximately  $9  billion.  These  numbers  reflect  significant  growth  over  the  last  decade  but  do  not 
indicate  the  changing  character  of  the  industry.  Maintenance  operations  are  being  recast  to  account  for 
the  introduction  of  new  and  more  complex  aircraft  and  the  use  of  more  sophisticated  maintenance  and 
inspection  procedures. 

Aviation  maintenance  is  in  fact  a 
large  industrial  system  which  includes 
many  elements  such  as  the  aircraft, 
the  maintenance  facility,  supervisory 
forces,  inspection  equipment,  repair 
equipment,  and  the  maintenance 
technician.  All  of  these  elements 
together  comprise  the  "maintenance 
system"  (Figure  6.1).  Within  this 
system,  the  technician  functions  and 
should  be  viewed  as  one  would  view 
any  other  element.  A  maintenance 
technician  has  a  set  of  operating  characteristics.  Conceptually  this  human  can  be  considered  in  essentially 
the  same  manner  as  other  system  elements  such  as,  for  example,  items  of  electronic  equipment.  The  major 
difference  is  that  the  human  is  significantly  more  complex  and  not  nearly  as  predictable.  However,  anyone 
responsible  for  designing  or  operating  a  system,  such  as  a  maintenance  system,  must  understand  the 
operating  characteristics  of  each  element  within  the  .system,  and  this  includes  the  human. 

Human  factors  is  a  discipline  which  seeks  to  understand  the  laws  of  human  behavior,  the  capabilities  and 
limits  of  humans,  and  the  effects  of  environmental  and  other  factors  on  human  behavior.  A  key  goal  of 
human  factors  is  to  provide  guidelines  for  the  optimum  use  of  humans  in  operating  systems.  An  allied 
goal  is  to  define  the  manner  in  which  variables  internal  and  external  to  a  system  affect  human  performance 
within  the  system. 


Modiantcs  employed 

ts 

SB.B19 

Maintenance  expenses 

= 

$8.8  billion 

(1 1.5%  of  operating  expenses) 

Major  carriers  contract  approximately  11%  of  maintenance  work 

Table  6.1  Maintenance  Parameters  for  U.S.  Scheduled 


Airlines  (1991  data)  ATA  (1992);  Office  of 
Technology  Assessment  (OTA)  (1988). 
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The  op)eraUon  of  any  system  can  only 
be  optimized  if  every  system  element 
is  working  properly  and  if  each 
element  is  carefully  coordinated  with 
every  other  element.  The  manager  of 
a  system  such  as  a  maintenance 
operation  should  have  all  necessary 
information  concerning  maintenance 
technicians  and,  in  particular,  those 
features  of  the  maintenance 
environment  which  serve  either  to 
enhance  or  to  degrade  technician 
performance.  The  manager  or 
supervisor  of  a  maintenance  activity 
can  be  aided  through  use  of  a  Human 
Factors  Guide  that  will  provide  this 
information  in  a  form  suitable  for  day- 
to-day  reference  use. 

A  Human  Factors  Guide  will  present  established  principles  of  job  design  and  work.  These  principles,  if 
well  applied,  can  make  a  major  contribution  toward  the  control  of  human  error  in  aircraft  maintenance  and 
inspection.  Issues  of  communications,  equipment  utilization,  work  scheduling  and  load,  work 
environment,  and  management  relations  all  are  of  importance  in  determining  worker  effectiveness.  A 
Human  Factors  Guide  should  cover  these  and  other  issues  of  human  performance  that  can  be  applied  in 
aviation  maintenance.  In  presenting  these  principles,  the  Guide  should  see  that  the  information  is 
especially  addressed  to  aviation  maintenance  and  inspection  needs. 

The  preparation  of  a  Human  Factors  Guide  is  timely  for  a  number  of  reasons.  The  most  important  of 
these  reasons  include; 

1.  There  is  a  need.  Safely  is  always  a  matter  of  concern.  The  Guide  can  contribute  to  maintenance 
efficiency  and  to  the  control  of  human  error  in  maintenance.  This  in  turn  will  support  continuing 
safety.  There  also  is  the  matter  of  cost  control.  Maintenance  effectiveness  contributes  to  cost 
reduction. 

2.  Human  factors  is  a  mature  and  growing  di.scipline.  The  knowledge  within  this  discipline  should  be 
used  to  support  maintenance  operations  in  the  same  manner  as  information  from  the  engineering 
sciences  support  specific  maintenance  procedures. 

3.  Considerable  information  concerning  human  factors  in  aviation  maintenance  has  been  developed  both 
through  the  research  conducted  by  the  FAA  Human  Factors  Team  and  through  the  conduct  of  human 
factors  meetings  during  which  beneficial  recommendalioas  have  been  made  by  repre.sentatives  of  air 
carrier  maintenance.  All  of  this  information  should  be  incorporated  into  a  Human  Factors  Guide. 

The  Human  Factors  Guide,  as  envisioned  now,  will  be  directed  primarily  toward  those  concerned  with  the 
development  and  operation  of  air  carrier  maintenance  organizations.  However,  the  Guide  will  be 
structured  to  meet  the  needs  al.so  of  a  larger  audience  interested  in  and  responsible  for  aviation 
maintenance.  In  general,  the  Guide  should  provide  human  factors  principles  and  data  of  u.se  to: 


Aircraft  for 
Inspection 
and  Repair 


Management 
Work  Communication 


Maintenance 
Tools  and 
Processes 


Data  Sources 


Maintenance  i 
Equipment 
and  Systems 


Figure  6.1  The  Maintenance  System 
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Maintenance  planners  and  supervisors 

Maintenance  inspectors  and  technicians 

FAA  management  and  FAA  inspectors 

Air  carrier  operators 

Designers  of  maintenance  equipment 

Aircraft  design  teams 

Aircraft  manufacturers 

6.1  DEVELOPMENT  OF  A  HUMAN  FACTORS  GUIDE 

The  development  of  a  Human  Factors  Guide  for  Aviation  Maintenance  is  underway.  The  first  step  in  this 
development  was  to  consider  the  premise  on  which  the  Guide  should  be  consU'ucted.  This  premise  is  that 
the  Guide,  or  any  such  document,  is  of  little  if  any  value  if  it  is  not  used.  The  aviation  maintenance 
community  must  use  the  Human  Factors  Guide  if  the  Guide  is  to  serve  any  real  purpose.  For  the  Guide 
to  be  used,  it  must  meet  ongoing  needs  of  maintenance  personnel  and  must  be  prepared  in  such  a  manner 
as  to  foster  use  by  this  group. 

In  order  to  collect  information  to  satisfy  the  above  requirements,  a  sampling  of  aviation  maintenance 
personnel  was  conducted.  The  information  solicited  was  designed  to  ensure  that  the  real  needs  of 
maintenance  personnel  would  be  met  and  that  the  Guide  could  be  consistent  with  the  ways  in  which  this 
sample  stated  they  were  likely  to  use  such  a  Guide. 

Approximately  60  individuals  affiliated  in  some  manner  with  the  air  carrier  maintenance  industry  were 
contacted  to  provide  guidance  on  sigm'ficant  maintenance  topics.  Names  were  selected  from  the  list  of 
attendees  at  earlier  FAA  Human  Factors  Meetings.  The  list  included  persons  both  from  the  United  States 
and  from  foreign  countries.  The  role  of  these  persons  in  aviation  maintenance,  ba.sed  on  their  replies,  is 
shown  in  Table  6,2.  The  fact  that  most  replies  were  received  from  "Inspection/  Maintenance  Managers" 
is  to  be  expected  since  this  job  category  constituted  the  bulk  of  the  initial  mailing. 

The  next  question  concerned 
the  value  users  would  place 
on  a  Human  Factors  Guide 
for  their  work.  Not 
surprisingly,  almost  all  of 
those  who  replied  indicated  a 
Human  Factors  Guide  would 
be  "very  valuable"  or 
"valuable."  Since  these 
replies  were  given  by  persons 
who  had  evidenced  interest  in 
this  topic  by  attending  human  factors  meetings,  these  replies  were  anticipated. 

The  individuals  were  questioned  on  the  anticipated  frequency  of  use  for  a  Human  Factors  Guide,  if  the 
Guide  contained  appropriate  information.  This  question  was  asked  in  order  to  determine  whether  the 
Guide  should  be  prepared  as  a  working  document  (as  a  job  aid)  or  as  a  reference  manual.  Table  6J 
presents  the  replies  to  this  question. 


Work  Classification 

Number 

Inspection/Maintenance  Manager 

22 

Educator/Trainer 

7 

Aircraft  Designer 

2 

Other  (Senior  Management, 

7 

Quality  Assurance.  Consultant.  Research. 

AMT  Associate.  Crew  Systems  Analyst) 

Table  6.2  Occupation  of  Respondents 
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The  replies  indicate  the  preferred  use  for  a  Human  Factors  Guide  would  be  as  a  working  document 
consulted  on  a  number  of  occasions  during  the  year. 

6.2  HUMAN  FACTORS 
COVERAGE 

The  coverage  provided  in  a 
Human  Factors  Guide  is  of 
great  importance  if  the  Guide 
is  to  be  truly  useful. 

Certainly,  the  topics  included 
in  this  Guide  should  be  those 
which  members  of  the 
maintenance  community 
consider  important.  In  order 
to  collect  information 
concerning  desired  coverage, 
an  outline  of  a  prototype 
Guide  was  prepared.  Each 
person  was  presented  a  list  of 
chapter  headings  from  the 
prototype  outline  and  asked  to 
judge  the  importance  of  the 
topic  on  a  five-step  scale 
ranging  from  "very  important" 

(weighting  of  five)  to  "not 
imfx)rtant"  (weighting  of  one). 

With  this  system,  had  each  of 
the  respondents  Judged  a  given 
topic  to  be  "very  important," 
that  topic  would  have  received 
a  total  score  of  190.  Results 
for  this  question  are  presented  in  Table  6.4.  While  there  is  a  dispersion  of  total  scores,  it  is  quite  apparent 
that  most  topics  were  judged  either  as  "very  important"  or  "important."  Tlie  topics  in  Table  6.4  arc  listed 
in  terms  of  decreasing  order  of  judged  importance. 

Prior  to  the  contact  with  the  60  individuals,  a  question  had  arisen  about  the  desirability  of  including  a 
section  within  the  Human  Factors  Guide  concerning  emotional  factors.  For  this  reason,  a  separate  question 
was  included  that  asked  "Should  the  Human  Factors  Guide  contain  a  .section,  not  usually  included  in  texts 
of  this  type,  that  addre.s.se.s  social  and  emotional  factors  that  can  affect  the  performance  of  a  worker?"  The 
following  responses  were  received: 

Yes  =  ?>2  =  6 

Obviously,  the  majority  of  the  respondents  believe  that  a  section  containing  the  above  information  should 
be  included  in  the  Guide. 


Weighted  Score 


Hurticin  Error  in  Maintenance 
Information  Exchange  and 
Communications 

Maintenance  Training  and  Practices 
Human  Capabilities  and  Limits 
Human  Performance 
Work  Requirements 
The  Maintenance  Workplace 
Job  Petlormance  Aids 
Man-Machine  Interface 
Workplace  Features 
Automation  in  Aircraft  Maintenance 


Table  6.4  Importance  of  Specific  Topics  for  Inclusion  in  a  Human 
Factors  Guide 


Table  6.3  Frequency  of  Anticipated  Use  for  a  Human  Factors 
Guide 
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To  ensure  that  no  appropriate  topics  were  missed,  each  respondent  was  asked  to  note  any  additional  topics 
believed  important  for  a  Guide  of  this  type.  Quite  a  few  replies  were  received;  most  appeared  to  be 
variants  of  the  topics  in  the  initial  list  presented  in  the  mailing.  However,  a  few  were  indeed  new  and  are 
listed  below: 

1.  Requalification,  limitations,  and  competency  verification  for  aviation  maintenance 
technicians. 

2.  Minimum  individual  qualifications  (eyesight,  color  blindness,  and  manual  dexterity)  for 
specific  maintenance  functions. 

3.  Sexual  harassment.  (This  could  become  increasingly  imprirtant  as  workforce 
demographics  change.) 

4.  Working  with  the  handicapped.  (The  recently  passed  Americans  With  Disabilities  Act 
gives  impetus  to  this  topic.) 

6.3  FORMAT 

For  a  Guide  to  be  useful,  it  not 
a  manner  designed  to  make  it 
easy  to  use.  Several  questions 
addressed  the  general  issue  of 
format.  The  first  question 
concerned  optimum  length. 

The  contacted  individuals  were 
asked  "To  be  most  usable, 
what  size  should  a  Human 
Factors  Guide  be?"  Table  6.5 
presents  the  responses. 

The  above  replies  clearly  point  to  a  shorter  rather  than  a  longer  Guide.  These  data  are  supported  by  a 
comment  submitted  by  one  respondent: 

A  Human  Factors  Guide 
should  be  50-75  pages 
for  handout  to  line 
management  personnel. 

It  should  be  100-300 
pages  for  managers  and 
supervisors  with 
decision  making 
capabilities  for 
resources  and  monies. 

The  next  item  asked  "What 
format  would  you  find  most 
useful?"  This  question  is 
considered  quite  important 


Format 


Number  of 
Selections 


Key  information  and  recommendations 
in  bullet  form,  with  illustrations 


Short  statements,  with  illustrations  8 

Running  prose,  with  illustrations  8 

Other  (Please  Specify)  2 

(Combine  short  statements,  with 
illustrations,  and  key  information  and 
recommendations  in  bullet  form,  with 
illustrations:  use  running  prose  -  segmented 
by  topic  statements;  use  electronic/digital 
_ format  with  key  word  search.) _ 

fable  6.6  Preferred  Format  for  a  Human  Factors  Guide 


only  must  contain  appropriate  information  but  also  must  be  presented  in 


Table  6.5  Desired  Length  of  a  Human  Factors  Guide 
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since  the  manner  in  which  information  is  presented  can  affect  the  extent  to  which  individuals  will  seek 
and  use  information  concerning  the  topic  being  presented.  Table  6.6  shows  the  results  for  this  question. 

Results  indicate  the  desired 
format  would  be  one  in  which 
information  is  presented  tersely 
and  concisely,  either  in  bullet 
form  or  using  short  statements. 

Illustrations  should  support  the 
materials  as  needed. 

A  question  next  was  asked 
which  relates  both  to  the 
length  of  the  Guide  and  the 
manner  in  which  materials  are 
presented.  Two  alternatives 
were  given  with  a  request  for  a 
shows  the  replies. 

These  results  show  a  slight  preference  for  a  shorter  Guide,  even  if  one  has  to  look  elsewhere  for  data 
supporting  and  elaborating  the  concise  information  presented  in  the  Guide.  One  supporting  comment 
illustrates  this:  "A  shorter  Guide.  Computerized  suppt>rting  data  would  be  very  nice." 

6.4  SUMMARY 

The  replies  of  the  maintenance  personnel,  combined  with  other  discussions  with  those  likely  to  use  a 
Human  Factors  Guide,  lead  to  the  following  conclu.sions  concerning  the  content  and  structure  of  the  Guide. 

Audience.  The  principal  users  of  a  Human  Factors  Guide  will  be  air  carrier  maintenance  planners  and 
sujxjrvisors.  However,  care  must  be  taken  that  the  structure  of  the  Guide  not  be  oriented  entirely  toward 
this  group.  A  Human  Factors  Guide  also  can  be  used  to  advantage  by  other  groups,  including  FAA 
management  and  FAA  inspectors  as  well  as  aircraft  design  teams  and  designers  of  maintenance  equipment. 
The  document  also  could  be  used  profitably  in  training  operations. 

Content.  Three  topics  have  been  identified  as  most  important  for  inclusion  in  a  Human  Factors  Guide. 
These  topics  are: 

•  Human  error  in  maintenance 

•  Information  exchange  and  communications 

•  Maintenance  training  and  practices 

All  topics  must  be  given  appropriate  coverage  in  the  Guide.  Greate.st  attention,  however,  will  be  given 
to  the  three  topics  listed  above. 

Size.  The  Human  Factors  Guide  should  not  be  a  large  d()cument  and  probably  should  not  exceed  2(K) 
pages  in  length.  A  larger  document  might  well  impact  u.se,  particularly  if  the  document  is  to  be  carried 


Alternatives 

Number 

A  shorter  guide,  presenting  brief 

21 

discussions  and  recommendations,  with 

supporting  data  elsewhere  (possibly 

in  another  book  or  in  a  computer 

data  base)? 

A  longer  guide,  with  supporting  data 

17 

included  as  appendices? 

Table  6,7  Preference  for  Physical  Structure  of  a  Human  Factors 
Guide 


preference  between  these  two.  Table  6.7  lists  the  two  alternatives  and 
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around  within  the  maintenance  facility.  Current  thinking  is  that  supporting  materials,  which  could  be  quite 
lengthy,  would  best  be  contained  in  a  computerized  data  base  in  a  CD-ROM  system.  With  proper  search 
strategies,  data  supporting  the  Guide  could  be  obtained  quite  rapidly. 

Style.  Information  within  the  Human  Factors  Guide,  such  as  basic  human  factors  principles  applied  to 
specific  maintenance  labor,  should  be  presented  concisely,  possibly  using  a  bullet  format,  with  supporting 
illustrations.  Introductory  chapters  and  materials  can  be  more  in  a  running  prose  form.  The  language 
should  be  simple  and  straightforward  English.  This  will  make  it  more  likely  that  the  message  is  conveyed 
as  intended.  Use  of  simple  English  also  will  help  should  the  Guide  be  translated  into  a  foreign  language 
for  use  in  overseas  maintenance  activities. 
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Chapter  Six  Appendix 
Sample  Section  Of  A  Human  Factors  Guide 

The  following  section  illustrates  the  manner  in  which  the  above  concepts  and  rules  would  be  applied  in 
the  preparation  of  a  section  for  the  Human  Factors  Guide.  The  section  is  presented  in  highly  abbreviated 
form  simply  to  show  the  appearance  and  general  content  of  a  part  of  the  Guide.  This  section  does  not 
indicate  the  depth  of  coverage  planned  for  individual  topics. 

1.0  SECTION  I:  AREA  AND  TASK  LIGHTING 

1.1  Importance  of  Lighting  in  Indus'rial  Operations 

Lighting  conditions  in  an  industrial 
workplace  are  important  both  for 
worker  productivity  and  for  worker 
comfort.  Numerous  studies  have 
examined  the  effect  on  worker 
productivity  of  varying  levels  of  task 
illumination  (see  Cushman,  1987).  In 
general,  these  studies  show  that 
performance  under  low  illumination 
improves  to  a  point  as  the  illumination 
level  is  increased.  Figure  1  shows  the 
reduced  time  required  to  complete  a 
typical  industrial  task  (reading  a 
micrometer)  as  the  level  of 
illumination  on  the  task  is  increased. 

Note  that  when  the  illumination 
reaches  about  1(X)  footcandles,  no 
additional  improvement  is  seen.  In 
general,  industrial  tasks  show  smaller 
and  smaller  improvement  in 
performance  as  illumination  is 
increased.  However,  the  point  where 
performance  finally  levels  off  is  task-depiendent.  Tasks  that  are  visually  difficult,  as  might  be  true  for 
inspection  activities,  will  require  more  light  to  achieve  be.st  performance  than  will  easier  tasks. 

Research  conducted  to  assess  the  effectiveness  of  illumination  on  performance  must  deal  with  two  issues 
which  can  affect  the  results.  First,  motivational  factors  must  be  controlled.  Subject,  or  workers,  who 
know  they  are  in  a  study  will  tend  to  perform  better  independent  of  the  illumination  level.  Second,  the 
age  of  subjects  is  important.  Workers  who  are  over  45  years 

of  age  will  show  more  improvement  with  increasing  illumination  than  will  younger  workers.  If  a  work 
group  contains  older  workers,  illumination  should  be  increased  to  account  for  this. 


Adapted  from  Sanders  and  McCormick,  1987. 
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Recommendations  for  proper  illumination  levels  for  various  activities  have  been  prepared  by  the 
Illuminating  Engineering  Society  and  are  presented  in  Table  1. 

Sanders  and  McCormick 
(1987)  point  out  problems  in 
arriving  at  recommendations 
for  adequate  illumination  to 
ensure  proper  task 
performance.  Interestingly, 
they  note  that  recommended 
levels  continually  increase 
through  the  years.  Current 
recommended  levels  are  about 
five  times  greater  than  levels 
recommended  30  years  ago  for 
the  same  tasks. 

Even  though  proper  levels  of 
illumination  are  provided,  task 
performance  can  be  degraded 
if  glare  sources  are  present. 

Glare  is  of  two  types.  Direct 
glare  is  produced  when  a 
bright  light  source  is  in  the  visual  field.  Indirect  glare,  often  called  reflected  glare,  is  reflected  from  the 
work  surface  and  reduces  the  apparent  contrast  of  task  materials.  Either  direct  or  indirect  glare  can 
degrade  task  performance.  Table  2  offers  suggestions  concerning  ways  to  control  the  effects  of  glare 
sources. 


Activity 

Level  (footcandles) 

Working  spaces  with  occasional 
visual  tasks 

10-20 

Performance  of  visual  tasks  of 
high  contrast  or  large  size 

20  -  50 

Performance  of  visual  tasks  of 
medium  contrast 

50  -  100 

Performance  of  visual  tasks  of 
low  contrast  or  very  small  size 

100-200 

Performance  of  visual  tasks  of 
low  contrast  and  very  small  size 
over  a  prolonged  period 

200  -  500 

Table  1  Recommended  Illuminance  Values  for  Different  Types  of 

Activity.  Adapted  from  Kantowitz  &  Sorkin,  1983. 


1.2  Lighting  Conditions  in  Aviation  Maintenance 

A  study  of  illumination  conditions  within  major  air  carriers  was  accomplished  as  part  of  an  FAA  audit 
(Thackray,  1992).  In  these  facilities,  overhead  lighting  typically  is  supplied  by  mercury  vapor,  metal 
halide,  or  high-pressure  sodium  lights.  The  principal  difference  here  is  in  terms  of  the  color  rendition  of 
the  lights.  While  color  rendition  is  probably  not  too  important  for  aircraft  exterior  maintenance  tasks,  the 
level  of  illumination  could  be.  Table  3  shows  average  illumination  levels  measured  at  different 
maintenance  work  areas,  both  for  day  shifts  and  night  shifts.  Table  3  also  presents  recommended 
illumination  levels  for  aircraft  repair  and  inspection  tasks.  Although  .slightly  below  recommended  levels, 
the  illumination  for  work  on  upper  and  lateral  surfaces  of  an  aircraft  appear  adequate.  For  repair  and 
inspection  conducted  below  wings,  the  fuselage,  and  within  cargo  and  engine  areas,  measured  illumination 
levels  are  not  adequate  and  supplemental  light  sources  are  required.  In  general,  supplemental  lighting  is 
provided  through  quartz  halogen  stand  lights,  dual  40-watt  fluore.scent  stand  fixtures,  single  hand-held 
fluorescent  lamps,  and  flashlights. 
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To  Control  Direct  Glare 

To  Control  Indirect  Glare 

Position  lighting  units  as  far  from  the  operator’s  line  of  sight 
as  practical 

Avoid  placing  lights  in  the  indirect-glare 

Use  several  low-intensity  lights  instead  of  one  bright  one 

Use  lights  with  diffusing  or  polarizing  lenses 

Use  lights  that  produce  a  batwing  light  distribution  and 
position  workers  so  that  the  highest  light  level  comes  from 
the  sides,  not  front  and  back 

Use  surfaces  that  diffusa  light,  such  as  flat  paint,  non-gloss 
paper,  and  textured  finishes 

Use  lights  with  louvers  or  prismatic  lenses 

Change  the  orientation  of  a  workplace,  task,  viewing  angle, 
or  viewing  direction  until  maximum  visibility  is  achieved 

Use  indirect  lighting 

Use  light  shields,  hoods  and  visors  at  the  workplace  if  other 
methods  are  impracticeU 

Table  2  Techniques  for  Controlling  Glare,  Adapted  from  Rodgers.  1987. 


Use  of  supplemental  lighting 
does  not  necessarily  solve 
existing  lighting  problems. 

The  FAA  audit  of  major 
carriers  found  that 
supplemental  lighting  systems 
frequently  were  placed  loo  far 
from  the  work  being 
performed  and  were  too  few 
in  number.  The  result  was 
that,  even  with  supplemental 
lighting,  the  illumination 
directly  at  the  work  site  was 
less  than  adequate. 

Aircraft  inspectors  generally 
use  small  flashlights  as 
supplementary  sources.  At 
times,  small  lights  mounted  on 
headbands  may  be  used.  The 
flashlights  provide 
illumination  ranging  from  1(X)  to  500  footcandles  and  are  acceptable  for  vi.sual  inspection.  However,  use 
of  the  flashlight  means  that  one  hand  cannot  be  u.sed  for  manipulation  of  the  systems  being  inspected. 

In  an  attempt  to  produce  more  even  lighting  within  maintenance  bays,  some  carriers  have  painted  tlie  walls 
and  even  the  floors  with  a  bright  white  reflective  paint.  While  this  does  tend  to  reduce  shadow  effects, 
other  problems  can  ari.se.  The  principal  one  is  glare.  Reflected  light  from  bright  .sources  produces  glare 
which  can  both  cau.se  discomfort  and  reduce  visibility  of  key  features  of  the  maintenance  task.  Tlie  glare 
tends  to  obscure  or  veil  part  of  the  vi.sual  task. 


Measured  (Footcandles) 

Day 

Night 

Hangar  area 

66 

51 

Below  wings,  fuselage  and  in  cargo  areas 

26 

15 

Within  fuselage 

23 

18 

Visual  inspection 

100-500 

(2  D-cell  flashlight) 

Recommended  (Footcandles) 

Min  Level 

Aircraft  repair,  general 

Aircraft  visual  inspection 

75 

Ordinary  area 

50 

Difficult 

100 

Highly  difficult 

200 

Table  3  Measured  Illumination  Levels  at  Major  Air  Carriers 


Compared  with  Recommended  Levels.  Adapted  from 
Thackray,  1990. 
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1.3  Guidelines 

The  goal  of  controlling  human  error  in  aviation  maintenance  requires  that  maintenance  be  conducted  under 
proper  lighting  conditions.  This  is  true  both  for  area  lighting,  that  which  illuminates  the  full  working  area, 
and  task  lighting,  that  directed  toward  specific  work  activities.  Improper  or  insufficient  lighting  can  lead 
to  mistakes  in  work  tasks  or  can  simply  increase  the  time  required  to  do  the  work.  In  a  program  directed 
toward  proper  lighting  conditions,  the  following  guidelines  should  be  observed: 

Guidelines 

•  Area  lighting  within  a  maintenance  facility  should  be  a  minimum  of  75  footcandles.  A 
level  of  100-150  footcandles  is  preferred. 

•  Care  must  be  exercised  to  see  that  the  light  level  available  for  night  maintenance  activities 
in  particular  does  not  drop  below  recommended  levels.  Any  lighting  studies  must  be 
conducted  both  during  the  day  and  at  night. 

•  Task  lighting  for  aircraft  inspection  requires  a  minimum  of  100  footcandles  of 
illumination.  For  difficult  inspections  or  fine  machine  work,  200-500  footcandles  of 
illumination  is  necessary. 

•  Supplemental  lighting  must  be  adequate  for  the  task  at  hand,  best  judged  by  the  worker. 
Task  lighting  should  be  placed  close  to  the  work  being  done  and,  if  feasible,  should  leave 
both  of  the  worker’s  hands  free  for  the  work.  If  systems  must  be  manipulated,  lights 
mounted  on  headbands  are  preferred  to  flashlights. 

•  If  the  workforce  contains  a  substantia!  percentage  of  older  workers,  i.e.  those  greater  than 
45  years  of  age,  recommended  lighting  levels  should  be  increased,  probably  on  the  order 
of  50  percent. 

•  Glare  sources  should  be  controlled.  Supplemental  lighting  should  be  placed  as  far  from 
a  worker’s  line  of  sight  as  practical.  Reflected  glare  can  be  changed  by  reorienting  the 
work  surface  or  changing  the  iX)sition  of  lights.  Worker  complaints  are  the  best  means 
for  identifying  offending  glare  sources. 

1.4  Procedures  for  Evaluating  Light  Conditions 

The  best  procedure  for  determining  if  lighting  conditions  are  adequate  is  through  the  services  of  either  the 
industrial  hygiene  department  or  the  safety  department  of  the  air  canier.  Individuals  in  these  departments 
typically  are  trained  in  procedures  for  conducting  an  environmental  audit,  possess  the  necessary 
measurement  equipment,  and  understand  the  problems  involved  in  obtaining  meaningful  measurements. 
Specialists  from  these  departments  al.so  will  be  able  to  provide  a  proper  evaluation  of  the  audit  results. 

If  the  services  of  specialists  are  not  available,  maintenance  managers  can  as.sess  lighting  conditions 
themselves.  Photometric  equipment  is  available  which  will  priJvidc  accurate  (generally  plus  or  minus  five 
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percent)  measurement  of  facility  lighting.  Illuminometers/photometers  are  available  commercially  for  a 
price  in  the  order  of  $1,000.  Catalogs  of  scientific  equipment  describe  these  items. 
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Chapter  Seven 

The  Effects  of  Crew  Resource  Management  (CRM)  Training  in  Maintenance: 

An  Early  Demonstration  of  Training  Effects  on  Attitudes  and  Performance 

7.0  INTRODUCTION 

There  are  encouraging  results  from  initial  data  on  the  effects  of  a  CRM  training  program  for  technical 
operations  managers  on  attitudes  and  maintenance  performance. 

The  study  is  intended  as  an  illustration  of  results  beginning  to  emerge  after  only  a  few  months  experience 
in  a  long  term  program  of  effective  communication  to  improve  safe,  dependable  and  efficient  performance. 
Specifically,  the  analyses  reported  here  compare  managers’  pre-  and  post-training  attitudes  about  a  variety 
of  management  and  organizational  factors  as  well  as  pre-  and  post-performance  measures  in  several 
maintenance  (and  related)  departments.  Some  highlights  are  as  follows: 

1)  Participants’  immediate  response  to  the  training  was  very  prisiiive.  They  were  even  more  positive 
than  those  from  other  studies  investigating  CRM  in  maintenance  or  in  flight  operations. 

2)  Changes  in  relevant  attitudes  measured  immediately  before  and  after  training  reveal  strong  and 
positive  changes  following  training,  for  three  of  the  four  indices  measured.  Follow-up  results 
several  months  after  training,  reveal  that  these  changes  are  stable;  forthcoming  data  should 
continue  to  strengthen  these  conclusions. 

3)  Most  maintenance  performance  measured  before  and  after  the  CRM  training  sessions,  examined 
in  the  present  paper,  show  significant  changes  in  the  expected  direction,  indicating  a  positive 
effect  of  training. 

4)  A  pattern  of  relationships  in  the  expxjcted  direction  were  observed  between  the  post-training 
attitudes  and  the  post-training  performance.  In  particular,  those  measures  predicted  (by  the 
program’s  trainers  and  managers)  to  be  particularly  seasitive  to  the  effects  of  the  CRM  uaining 
were  affected. 

5)  Finally,  in  follow-up  surveys',  managers  responses  indicate  that,  when  transferring  skills  and 
knowledge,  they  tend  to  initiate  interactive  behaviors  instead  of  passive  ones.  Anecdotal  evidence 
is  also  beginning  to  confirm  these  changes  and  their  positive  impact  on  performance. 

7.1  THE  EFFECTIVE-COMMUNICATION  PROGRAM:  CREW  RESOURCE 
MANAGEMENT  TRAINING 

Resource  management  Gaining  for  airline  fiight  crews  was  introduced  in  the  late  1970’s  (Helmreich.  1979). 
It  has  spread  to  many  air  carriers  in  the  U.S.  commercial  aviation  industry,  to  .several  foreign  carriers  and 
to  various  sectors  of  U.S.  and  Canadian  military  aviation.  That  training  has  been  extended  from  the 


“All  surveys  and  questionnaires  were  distributed  by  the  airline  to  airline  employees.  The  survey 
instruments  were  not  developed  or  distributed  by  the  FAA. 
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cockpit  to  cabin  crews,  to  maintenance  teams  and  to  air  traffic  centers  and  is  now  referred  to  as  Crew 
Resource  Management  (CRM)  training.  Although  specific  programs  differ  from  one  organization  to 
another,  Crew  Resource  Management  typically  involves  training  in  several  team-related  concepts: 
communication  skills,  self-knowledge,  situational  awareness,  and  assertiveness  skills. 

The  effect  of  Crew  Resource  Management  training  in  airline  flight  operations  has  been  widely  studied 
during  the  1980s.  Numerous  reports  document  CRM’s  positive  impact  on  the  attitudes  and  performance 
of  flight  crews  (cf.,  Helmreich,  Foushee,  Benson,  &  Russini,  1986;  Helmreich,  Predmore,  Irwin,  Butler, 
Taggart,  Willhelm,  Clothier,  1991).  Taken  together,  the  evidence  shows  that  team  coordination  among 
aviation  "managers,"  and  between  them  and  subordinates,  improves  system  effectiveness  and  safety. 

As  a  result  of  recent  work  researching  team  concepts  in  aviation  maintenance,  further  investigations  have 
been  recommended  by  both  industry  and  government  groups  as  a  national  priority  (Federal  Aviation 
Administration,  "The  National  Plan  for  Aviation  Human  Factors,"  Washington,  DC:  1991).  A  first 
instance  of  an  airline  applying  CRM  to  maintenance  operations  was  reported  by  Taggart  (1990).  Others 
are  in  beginning  or  planning  stages.  This  report  describes  the  case  of  a  second  company’s  program  to 
apply  CRM-type  training  in  maintenance. 

7.2  THE  PRESENT  STUDY 

The  analyses  reported  below  will  assess  the  relationships  among  managers’  pre-  and  post-training  attitudes 
about  a  variety  of  management  and  organizational  factors  and  the  levels  of  maintenance  performance 
measures  in  a  large  U.S.  airline  (hereinafter  called  "the  company"). 

7.2.1  The  Purpose  of  the  Program  and  the  Course 

The  program’s  champion  is  the  company’s  Senior  Vice  President  for  Technical  Operations.  He  has  stated 
that  his  aim  for  the  training  and  evaluation  program  is  to  improve  human  resource  (HR)  management 
using  science-based  tools  and  techniques  for  diffusion  and  evaluation.  Further,  the  effectiveness  of  this 
training,  as  measured  by  the  ongoing  evaluation  of  it,  can  help  to  direct  the  industry’s  HR  practices  in  the 
future;  and  to  guide  the  development  of  future  ATA  and  FAA  training  policies  and  regulations.  The 
training  is  a  technical  operations  program  entirely.  It  is  managed  and  administered  by  technical  operations 
people,  and  the  trainers  (assisted  by  professional  communications  training  consultants)  are  technical 
operations  people  too.  An  Editor  from  Aviation  Week  &  Space  Technology  participated  in  the  training 
at  the  invitation  of  the  company.  The  two  articles  he  wrote  (Fotos,  1991)  further  describe  top 
management’s  reasons  for  undertaking  the  program,  and  provide  impressions  of  the  training  itself. 

Course  objectives.  The  purpose  of  the  training,  as  stated  by  trainers  on  the  first  day  of  each  haining 
session,  is  "To  equip  all  Technical  Operations  personnel  [management  first]  with  the  skill  to  use  all 
resources  to  improve  safety  and  efficiency."  The  objectives  (the  more  specific  goals  of  the  training)  are 
also  clearly  stated  during  the  trainers’  introductory  remarks.  They  are  as  follows: 

1.  Diagnose  organizational  "norms"  and  their  effect  on  safety. 

2.  Promote  assertive  behavior. 

3.  Understand  individual  leadership  styles. 

4.  Understand  and  manage  stress. 
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5.  Enhance  rational  problem  solving  and  decision  making  skills. 

6.  Enhance  interpersonal  skills 

The  course  as  designed  for  the  objectives.  The  aims  and  objectives  of  the  training  are  facilitated  through 
a  course  syllabus  containing  12  modules  (Appendix  A  contains  the  current  syllabus). 

State  of  training  completed.  To  date,  less  than  half  of  the  technical  maintenance  operations  stations, 
departments,  or  functional  divisions  in  the  company  have  been  able  to  send  alt  of  their  managers  to  the 
training  -  this  can  only  be  done  over  time  to  avoid  the  absence  of  all  managers  and  supervisors,  at  the 
same  time,  from  their  departments  or  stations  while  training  is  conducted.  Thus,  the  present  analysis  uses 
data  from  roughly  one-quarter  of  the  whole  technical  operations  management  staff  who  will  eventually 
complete  the  training.  Overall,  however,  most  cities  among  the  company’s  31  locations  have  had  at  least 
one-third  of  their  maintenance  managers  attend  a  session.  By  the  end  of  1991,  the  majority  of  people 
remaining  to  complete  the  course  are  located  in  the  company’s  three  largest  cities  -  they  comprise  mainly 
maintenance  supervisors  (over  50%  of  whom  still  need  to  attend);  and  assistant  supervisors  (nearly  75% 
who  have  yet  to  attend). 

Maintenance  work  units  as  the  focus  of  the  analyses.  Tlie  analyses  described  in  this  report  are  intended 
to  illustrate  the  effect  of  changes  in  work-unit  attitudes  (data  collected  from  individual  managers  who 
underwent  the  training,  combined  into  averages  for  the  units  to  which  they  belong)  and  the  performance 
data  (classified  into  safety,  dependability,  and  efficiency  categories)  constructed  into  measures  for  the  same 
work  units. 

How  is  the  course 
experienced? 

Summary  results  of 
answers  to  a  post- 
training  question 
concerning  the 
perceived  usefulness 
of  the  CRM  training 
are  shown  in  Figure 
7.1.  This  figure 
compares  the  sample 
from  the  present  study 
with  two  flight 
operations  samples 
from  other  companies 
(Helmreich,  1989)  and 
with  one  other 
technical  operations 

sample  (Taggart,  1990).  Like  the  other  maintenance  department  measured,  the  data  from  the  Technical 
Operations  division  of  the  present  company  reveals  high  enthusiasm  about  the  training.  None  of  the 
respondents  in  either  maintenance  department  sample  said  that  the  CRM  training  was  either  a  "waste  of 
time"  or  only  "slightly  u.seful."  In  addition,  a  very  high  percentage  (687fi)  of  the  pre.sent  company  found 
the  training  to  be  "extremely  u.seful."  This  training  appears  to  be  very  well  received  by  its  "customers." 
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Figure  7.1  Ratings  of  U.sefuiness  of  CRM  Training  for  Flight  Operations  and 
Technical  Operations 
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7.3  CREW  RESOURCE  MANAGEMENTAECHNICAL  OPERATIONS  QUESTIONNAIRE 

7.3.1  Background  Related  to  Measurement  of  CRM  Training 

A  questionnaire  called  "CMAQ"  (for  Cockpit  Management  Attitudes  Questionnaire)  has  long  been  a 
recognized  measure  for  assessing  flight  crew  attitudes  (Helmreich  et  al..  1986).  The  CMAQ  contains  25 
items  measuring  attitudes  that  are  either  conceptually  or  empirically  related  to  CRM.  Taggart  (1990) 
revised  the  CMAQ  for  use  in  a  technical  operations  department,  and  reported  positive  initial  results 
following  CRM  training. 

A  recent  study  involved  the  analysis  of  the  CMAQ  instrument  through  the  use  of  Factor  Analysis,  a 
technique  to  explore  for  a  consistent  internal  structure  (Gregorich  et  al.,  1990).  In  their  study  these 
authors  showed  the  relationships  among  the  25  CMAQ  items  clustered  into  the  following  four 
constellations  of  attitudes;  Sharing  Command  Responsibility,  Value  of  Communication  and  Coordination, 
Recognizing  Stressor  Effects,  and  Avoiding  Conflict. 

Because  multiple  items  tap  into  specific  aspects  of  CRM  (e.g.,  "communication."  and  "interpersonal 
skills"),  Gregorich  et  al..  (1990)  combined  the  items  into  composite  indices.  Such  index  scales  permit 
more  detailed  assessment  of  separate  but  related  attitudes  than  a  single  total  score  for  the  entire 
questionnaire,  and  they  also  provide  more  accurate  and  reliable  results  than  are  available  from  each  of  the 
individual  questionnaire  items  atone. 

7.3.2  Survey  Used  in  the  Present  Study 

Measurement  of  attitudes.  The  "Crew  Resources  Managemeni/Technical  Operations  Questionnaire" 
(CRM/TOQ)  as  used  in  the  present  study  is  a  modified  version  of  Taggart’s  (1990)  revised  CMAQ. 

The  CRM/TOQ  contains  26  multiple  response  items.  The  company’s  modifications  of  the  CMAQ 
involved  removing  five  questions  and  adding  six  others.  The  five  questions  were  removed  becau.se  they 
either  lacked  predictive  validity  (as  reported  by  earlier  flight  crew  studies;  Helmreich  et  al.,  1986)  or,  in 
the  company’s  opinion,  lacked  relevance  to  technical  operations. 

Six  questions  were  al.so  added  to  the  CRM/TOQ.  based  on  items  intended  to  measure  respondents’ 
perceptions  of  behaviors  dealing  with  attainment  of  work  goals  (Geirland  &  Cotter,  1990). 

Measurement  over  time.  At  the  time  of  this  writing  (some  six  months  into  an  18  month  training  process) 
the  data  sets  are  necessarily  incomplete.  Over  500  of  1.8(K)  total  managers  have  already  attended  the  two- 
day  training  program.  Virtually  all  tho.se  attending  the  training  have  completed  the  CRM/TOQ  both  pre- 
and  post-training.  Some  4(X)  of  the  5(K)  pre-  and  post-training  CRM/TOQ  completed  have  been  entered 
into  the  data  base  and  will  be  used  in  the  present  demonstration.  It  is  still  too  early  in  the  training  and 
evaluation  process  to  have  abundant  data  for  the  CRM/TOQ  follow-up  (two  and  six  month),  or  many 
months  of  performance  data  subsequent  to  the  training;  but  the  present  report  will  describe  the  results  of 
some  60  two-month  and  50  six-month  follow-up  que.stionnaires  returned  so  far.  This  follow-up  version 
of  the  CRM/TOQ  was  mailed  in  early  October,  1991,  to  the  homes  of  those  managers  who  had  attended 
the  CRM  training  in  May  or  June  for  the  "six-month"  measure,  while  attendance  during  July  and  August 
qualified  for  the  two-month  follow-up. 
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Collecting  baseline  data  for  testing  statistical  goodness  of  data.  During  the  last  week  in  May  1991  and 
before  the  training  program  began,  the  CRM/TOQ  was  sent  to  the  homes  of  all  directors,  managers, 
supervisors,  and  assistant  supervisors  (1,787  total).  Within  five  weeks  over  9(X)  questionnaires  had  been 
returned  for  a  return  rate  of  over  50%.  A  return  rate  this  high  is  considered  quite  acceptable  with  paper 
and  pencil  surveys  of  this  type  (Borg  &  Gall,  1986)  —  especially  since  a  reminder  or  prompt  was  not 
possible,  given  that  the  training  program  was  due  to  be  announced  and  start  within  two  weeks  of  the 
mailing.  This  questionnaire  was  termed  the  "baseline"  survey.  A  sample  of  the  "baseline"  questionnaire 
and  one  of  the  "follow-up"  questionnaires  are  included  here  in  Appendix  B  and  Appendix  C,  respectively. 

The  mean  scores  of 
the  baseline  results 
could  reveal  bias  with 
a  50%  return,  but 
continuing 
comparisons  between 
those  baseline  data  and 
immediate  pre-training 
results  (where  very 
close  to  1(X)%  return 
rates  are  realized) 
similar  relative  levels 
for  the  six  composite 
index  scales  are 
revealed.  Figure  7.2 
presents  the 
comparison  profiles. 

These  results  show 
few  differences  in  absolute  scores  and  similar  profiles. 

7.4  PERFORMANCE  DATA  DESCRIBED 

Technical  Operations  managers  in  the  company  already  collect  performance  data  in  abundance.  Table 
7.1  presents  the  14  measures  used  as  end-result  criteria  in  the  present  study.  Three  conditions  were  met 
in  order  to  include  these  measures  in  the  work-unit  analysis  reported  here.  First  (and  obviously)  the 
performance  measures  need  to  be  available  by  work  unit,  and  not  just  by  department  or  function.  Second 
the  measures  must  be  ones  that  people  in  the  work  unit  can  affect  by  their  actions  and  not  merely  ones 
that  are  conveniently  assigned  to  a  unit  -  but  for  which  it  can  do  little.  The  third  condition  applied  was 
that  the  measures  not  be  directly  related  to  or  completely  determined  by,  other  measures  in  the  set. 

The  14  measures  are  classified  into  three  performance  categories:  Safety,  Dependability  and  Efficiency; 
and  this  cla.ssification  is  shown  in  Table  7.1.  The  trainers  and  administrators  of  the  CRM  course 
evaluated  the  14  performance  measures  and  predicted  which  of  them  would  be  more  sensitive  to  effects 
of  CRM  training.  Their  conclusions  were  that  six  measures  were  the  most  readily  improved  by  the 
training.  The.se  six  performance  measures  included  both  aircraft  safety  items  (ground  damage  and 
turnbacks),  days  lost  to  occupational  injury,  dependability  based  on  departures  within  5  and  15  minutes 
and  delays  due  to  maintenance  error.  Their  .second  ranked  performance  measures  likely  to  improve  in 


Figure  7.2  Mean  Scores  of  Indexes  for  Baseline  and  Pre-Seminar  for 
Technical  Operations 
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response  to  training  included  sick  days  lost,  departures  within  60  minutes,  canceled  llights  due  to  late  from 
maintenance,  base  maintenance  performance  to  plan,  hours  applied  to  production  and  overtime  paid.  The 
performance  indicators  rated  least  likely  to  improve  as  a  function  of  the  training  were  departures  later  than 
60  minutes  and  warehouse  parts  service  levels. 


I.  MEASURES  OF  SAFETY 

A.  Safety  of  Aircraft 

1 .  Number  of  Ground  Damage  Incidents*  (44) 

2.  Number  of  Air  Tumbacks/Oiversions  caused  by  human  error*"  (31) 

B.  Personal  Safety 

3.  %  Occupational  Injury  Days  Lost*  (60) 

4.  %  Sick  Days  Lost*  (60) 

II.  MEASURES  OF  OEPENOABIUTY 

A.  Departure  Performance  (Line  Station  data  only,  n=31) 

5.  %  Departure  within  5  minutes* 

6.  %  Departure  within  1 5  minutes* 

7.  %  Departure  within  60  minutes* 

8.  %  Departure  over  60  minutes,  but  not  canceled" 

9.  Number  of  Delays  due  to  late  from  Maintenance*"  (n=35) 

B.  Service  Levels  (Materials  Sen/ices  only,  n-9) 

10.  %  Rotable  parts  available" 

11. %  Expendable  parts  available" 

C.  Base  performance  to  plan  (n=4) 

12.  %  Heavy  Checks  on  time  to  initial  plan* 

III.  MEASURES  OF  EFFICIENCY 

A.  Productivity 

13.  Ratio  of  Hours  paid  to  those  applied  to  production*  (7) 

B.  Cost 

14.  %  Overtime  paid  to  total  wage  bill*  (60) 


(maximum  number  of  work  units  possible  are  in  parentheses) 


as  Measures  evaluated  most  sensitive  to  effects  of  CRM  training 
bs  Measures  evaluated  moderately  sensitive  to  effects  of  CRM  training 
cs  Measures  evaluated  least  sensitive  to  effects  of  CRM  training 
ds  These  measures  exhibit  skewness  and  lack  of  variability 


Table  7.1  Technical  Operations  Performance  Measures  Available  by  Work  Unit 

The  present  analysis  employs  four  months  of  performance  data.  June  and  July  1991  are  considered  "pre¬ 
training,"  since  they  are  coincident  with,  or  precede,  mo.st  of  the  training  .sessions  measured  here.  August 
and  September  are  used  as  the  post-training  measures.  The.se  performance  data  are  available  for  .^1  line 
maintenance  stations,  4  base  maintenance  stations,  ?>  .shops,  9  materials  services  warehouses,  and  1.^ 
inspection/quality  assurance  units,  for  a  maximum  of  60  data  points  each  month.  Tlie  planning  and 
engineering  units  in  technical  operations  do  not  yet  have  unit  performance  data  which  they  can  be 
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measured  against,  although  the  company  is  developing  such  measures  lor  inclusion  in  later  analyses.  As 
noted  above,  the  attitudinal  data  are  not  yet  available  for  all  60  work  units;  and  in  some  cases  performance 
data  is  unavailable  for  certain  work  units  for  the  four  months  analyzed  for  this  report.  Becau.se  of  this, 
the  full  number  of  maintenance  units  available  for  the  present  analysis  is  somewhat  le.ss  than  the  full  .set. 

7.5  THE  ANALYSIS  PLAN  DESCRIBED 


7.5.1  A  Model  for  Testing  Relationships 

The  theory  tested  is  that  training  in  teamwork  and  communications  inlluences  maintenance  personnel 
attitudes  and  perceptions,  and  these  in  turn  produce  positive  changes  in  group  behavii)r  which  impact 
performance.  The  analysis  of  the  model,  specified  in  Figure  7.3.  tests  these  inlluences  by  postulating 
systematic  relations  among  a  .set  of  explanatory  variables.  Figure  7.3  provides  a  diagram  of  the 
suppo.sition  or  small  "theory"  examined  in  this  model. 

The  conclusive 
a.s.sessmenl  will  be  a 
causal  analysis  proving 
the  strength  of 
inlluence  the  training 
has  on  subsequent 
safety,  dependability 
and  efficiency.  The 
pre.sent  data  do  not  yet 
establish  true  causal 
linkages.  However, 
the  pre.sent  data  do 
show  associations 
between  pre-  and  po.st- 
training  attitudes  and 
perceptions  (measured 
by  the  CRM/TOQ). 
and  between  p»)st- 
training  CRM/T()(^ 
results  and 
performance. 


2-Mo«lh  Follow-up 
Quetliofwvaiie 


Pia-<fainino 

Qua«liorMaiia 


4.5.6 


6-Monlh  FoMoiv-iip 
^  QuctlionnAMe  - 


12‘Monlh  Foio«v-up 
- Quetlionnaiie  " 


Pie-(iainin0 
PaifcN  nance 


Aiioyi  Legend 

PiedicUvc  telalionsNpj  tejied 
^  To  be  lesied  as  data  become  available 


PeilofmdfM:e 


^  Pott  Ir aiming 
Peifoinance 


t 

^  Posl-lraining 
^  Peifoinence ' 


Numbef  Legend 
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1  W4coMonTe$t  4  Discfinwani  Analysis 

2  Repeatedmeasues  ANOVA  5  Speaiman  Rank  Coiraialion 

3  Mtilrvanale  ANOVA  6  Miilfie  Regiession 


Figure  7.3  A  Model  of  CRM  Training,  Attitude  Change,  and  Technical 
Operations  I’erformance 


7.6  RESULTS  OF  STATISTICAL  ANALYSIS 

7.6.1  Factor  Analysis  of  Survey  Items 

Among  other  statistical  priKcdures  undertaken  to  a.s.sess  the  goodness  of  the  CRM/T()(^  measure,  two 
Factor  Analyses  were  performed,  using  data  from  the  9(K)-plus  completed  questionnaires  from  the 
"baseline"  survey.  The  first  Factor  Analysis  was  conducted  to  determine  the  underlying  internal  structure 
among  the  answers  to  the  20  attitude  que.stions,  and  the  .second  was  run  for  the  six  iiuestions  that  measured 
perceptions  of  behaviors  dealing  with  atiainment  of  work  goals. 


Chapter  Seven 

First,  the  structure  of  the  attitude  data  was  found  to  be  very  similar  to  the  structure  reported  by  Gregorich 
et  al.  (1990).  On  the  strength  of  this  similarity,  it  was  decided  to  apply  the  same  names  to  the  first  three 
of  the  four  CRM/TOQ  factors  that  Gregorich  applied  to  the  CMAQ.  Because  ah  factors  derived  from  the 
revised  CMAQ  were  .statistically  strong,  even  the  "avoiding  conllict"  composite  (not  subsequently  used 
by  Gregorich,  et.  al.,  (1990))  was  retained  here  as  the  reflected  index  "Willingness  to  Voice 
Disagreement."  The  item  sums  for  each  factor  were  averaged  to  form  four  index  scores  for  each 
respondent.  The  four  factors  were  titled  (1)  Command  Responsibility,  (2)  Communication  and 
Coordination,  O)  Recognition  of  Stressor  FTfects,  and  (4)  Willingness  to  Voice  Disagreement.  The 
reliabilities  were  good  for  scales  of  this  length,  ranging  between  .54  and  .56  for  Indices  1,  2,  and  4.  The 
reliability  coefficient  of  .39  for  Index  3  was  lower  than  desired  for  confidence  in  a  stable  index,  but  the 
association  among  the  contributing  items  "communalities"  were  rea.sonabty  high  at  .58  and  .59. 

Secondly,  items  measuring  attainment  of  work  goals  loaded  on  two  separate  factors.  One  factor  consisted 
of  items  a.s.sessing  "Goal  Attainment  with  My  Group  and  the  other  "Goal  Attainment  with  Other  Groups;" 
their  respective  reliability  coefficients  were  .77  and  .74.  Two  additional  indices  were  formed  based  upon 
the.se  results. 

7.6.2  Attitude  Changes  Over  Time 

Pre-  and  rn^st-training  .scores.  The  "before"  and  "after"  scores  for  the  four  compr)site  attitude  .scales  were 
compared  and  scores  of  the  385  individual  managers  (who.se  pre-  and  po.st-  questionnaires  could  be 
matched)  combined  and  averaged  by  the  work  units  they  belong  it).  Several  analy.ses  allow  comparison 
of  pre-training  attitudes,  to  post-training  altitudes,  as  measured  by  the  CRM/TOQ.  Both  the  Wilcoxon 
Matched-Pairs  Signed-Ranks  Test  and  the  Repeated-Measures  Analysis  of  Variance  tests  were  used  to 
assess  differences  between  pre-  and  post-measures  and  each  te.sl  has  its  advantages,  depending  upon  the 
nature  of  the  data  .set 
(SPSS  -  U.ser’s  Guide, 

1990).  In  addition. 

Multivariate  Analysis 
of  Variance 
(MAN  OVA)  was 
employed  to  lest  the 
effects,  across  time,  of 
demographic  variables, 
such  as  job  title,  job 
tenure  or  age,  on 
attitudes  and  of 
different  departments 
on  performance 
measures.  Figure  7.4 
.shows  the  comparison 
of  overall  averages  for 
the  four  compo.sile 
attitude  scales. 


Figure  7.4  Mean  .Scores  of  Indexes  for  l*re-  and  Post-.Seminar  Technical 
Operations 
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All  of  the  pre-training  and  post-training  changes  are  in  the  expected  direction  except  for  "Willingness  to 
Voice  Disagreement,"  which  shows  a  shift  away  from  voicing  disagreement  and  toward  avoiding  conflict. 
All  of  the  differences  are  significant  at  the  .05  level  or  belter. 

Willingness  to  Voice  Disagreement:  Effect  of  Age  and  Job.  Figures  7,5  and  7.6  pre.sent  the  change  in 
"Willingness  to  Voice  Disagreement"  for  the  age  of  the  maintenance  personnel  and  for  the  given 
categories  of  job  classifications,  respectively.  Each  of  these  categories  interacted  with  the  repeated 
variable  (pre-post  survey),  affecting  the  magnihide  of  negative  shift  (Pre-Post  x  Age;  F=1.96,  p  <.10;  l\e- 
Post  X  Job:  F=5.20,  p  <.01). 


These  interactions  are 
interesting  because 
they  further  explain 
the  negative  shift  in 
attitude  on  the 
"Willingness  to  Voice 
Disagreement"  scale. 

Figure  7.5  shows  that 
the  oldest  and 
youngest  members  of 
the  sample  exhibit  the 
greatest  negative  shift, 
whereas  the  remainder 
show  very  little 
change  in  "Willingness 
to  Voice 
Disagreement" 
immediately  following 
the  training.  In 
addition.  Figure  7.6 
shows  that  the 
assistant  supervi.sors 
(who  are  over¬ 
represented  by  the 
youngest  and  oldest 
members  of  the 
sample)  and  managers 
have  the  greatest 
negative  shift  in 
attitude. 


Figure  7.5  Mean  Scores  for  I’re-  and  Post-Seminar  "Willingne.ss  to  Voice 
Disagreement"  by  Years  of  Age 


Figure  7.6  Mean  Scores  for  lYe-  and  Post-Seminar  "Willingness  to  Voice 
Disagreement"  by  Job  Title 
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Recognition  ol  Slrcssor  HITccls:  HUccIs  ol  Dcparlincni.  The  magnitude  ol  change  in  altitudes  on  the 
index  "Recognition  of  Stressor  Kneels"  were  affected  by  the  department  alillialion  of  the  respondent. 


revealing  a  signiricani 
interaction  between 
attitude  change  and 
department  (1‘re-Rost 
X  Dept:  K=2.25, 
p<.()5).  Figure  7.7 
shows  that  the 
altitudes  of  base 
maintenance  and 
quality  departments 
change  little  over  time 
whereas  those  of  line 
maintenance  and 
planning  departments 
improve  considerably. 

(ioal  Allainmenl.  The 
.scores  for  two  goal 
attainment  scales  can 
also  be  compared 
before  and  after 
training.  Since  lhe.se 


Figure  7.7  Mean  .Scores,  Rre-  and  Post-Training,  for  "Recognition  of 
Stressor  Kffecls"  by  Department 


are  perceptions  of  behavior,  and  not  measures  of  altitudes,  goal  attainment  scales  were  initially  not 


expected  to  show  changes  because  there  would  not  be  lime  to  change  the  behaviors  which  were  the  focus 


of  the  measures.  Because  perceptions  are  not  behavior,  like  altitudes,  they  can  be  more  quickly  inlluenced 


by  exposure  to  new 
information,  or  by 
reconsidering  initial 
assessments.  One 
statistically  signilicani 
change  was  found  in  the 
goal  attainment  with 
one’s  group  immediately 
following  training. 

Figure  7.X  shows  the 
pre-post  averages  for 
both  of  these  measures. 

It  is  interesting  to  note 
the  direction  of  changes 

in  Figure  7.8.  Por  the  Figure  7.8  Mean  Scores  Pre  and  Posl-.Seminar  lor  (ioal  Allainmenl  Indexes 
scale  focused  on 
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behaviors  to  achieve  goal  attainment  in  one’s  own  group  a  negative  change  occurred  (i.e.,  the  levels  ol 
discussion  and  encouragement  were  reported  to  be  higher  belore  the  training  than  afterwards).  This  might 
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be  explained  as  unrealistic  assessments  (i.e.,  over-optimistic  view  of  their  own  groups)  ol  tliis  group 
behavior  which  were  challenged  by  exposure  to  the  case  analyses  and  discussions  during  the  CKM 
training. 

The  slight  positive  shift  in  perception  of  activity  to  share  goals  with  other  groups’  approaches  .shows  a 
tendency,  but  is  not  significant  (p>  .1.5).  This  trend  may  al.so  be  explained  as  a  probable  effect  of  Gaining 
which  provided  new  information  (and  more  importantly  new  experiences  with  people  from  other 
departments  during  the  training)  about  the  similarity  of  goals  pursued  by  other  groups  and  functions  in 
technical  operations. 

Post-trainintt  attitudes  compared  with  two-  and  .six-month  follow-up.  Scores  for  die  four  indexes  were 
compared  with  results  from  the  follow-up  surveys.  The.se  results  compare  the  post-test  scores  with  the 
.50  or  41  individual  managers  whose  post-training  and  2-month  or  6-month  follow-up  questionnaires  could 
be  matched.  None  of  the  differences  are  significant  at  the  .05  level  or  better  (in  fact  none  approached 
significance  any  stronger  than  p  <15). 

Taken  together,  the 
results  show  that 
attitudes  about 
communication/ 
coordination  and 
sharing  command 
re.spoasibility  remain 
high  in  the  months 
following  CRM 
training.  The  goal 
attainment  measures 
were  not  included  in 
Figures  7.9  and  7.10 
and  no  .statistically 
significant  changes 

were  found  for  them  Figure  7.9  Mean  .Scores  of  Indexes  for  Post-Seminar  and  Two  Month 
cither.  Follow-up  for  Technical  Operations 

7.6J  Performance  Changes  Over  Time 

Pre-  and  post-trainini;  scores.  The  "before"  and  "after"  .scores  for  the  14  performance  measures  were 
compared  and  tested  for  statistical  significance.  Once  again  the  Wilcoxon  test,  the  Repeated-Measures 
analysis  and  the  Multivariate  Analysis  of  Variance  (MANOVA)  were  used.  Tliese  tests  represent  a  unit 
by  unit  comparison  of  the  scores  of  the  work  units  for  which  the  specific  measures  applied.  Many  of  the 
differences  are  .significant  at  the  .05  level  or  better,  but  some  are  in  the  opposite  direction  from  that 
expected  if  the  changes  arc  to  be  attributed  to  changes  from  the  CRM  training,  'fhe  next  four  figures 
(7.11,  7.12,  7.13  and  7.14)  pre.sent  the  overall  scores  averaged  over  all  the  work  units  lor  which  the 
performance  measures  apply.  The  .statistically  significant  differences  are  displayed  with  the  histograms 
in  the  figures  as  noted  below. 
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Measures  of  Safety 
Performance:  Safety  of 
Aircraft.  Figure  7.11 
presents  before-and 
after-training 
comparisons  of  two 
measures  of  aircraft 
safety  performance  - 
number  of  Ground- 
Damage  incidents  for 
44  work  units,  and 
number  of  Turnbacks 
or  Flight  Diversioas 
cau.sed  by  human  error 
for  31  line  stations. 

Both  of  these  Figure  7.10  Mean  Scores  of  Indexes  for  Post-Seminar  and  Six  Month  Follow- 
differences  are  in  the  up  for  Technical  Operations 

predicted  direction  and 

significant  (p<.()5).  Although  there  are  ample  alternative  explanations  for  these  improvements  between 
June/July  and  August/September,  it  is  important  to  note  that  CRM  training  could  be  responsible  for  at  least 
some  of  the  ob.served 
changes.  The  trainers 
and  training 
administrators  had 
predicted  that  these 
two  safety  measures 
would  be  among  the 
most  likely  to  improve 
because  of  the 
training. 


Mea.sures  of  Safety 


Safety.  Figure  7.12 
pre.sents  comparisons 
for  the  two  measures 
percent  Days  Lost  to 
Occupational  Injury, 

and  percent  Sick  Days  Lost  for  the  55  and  .*>4  work  units  respectively  for  which  data  were  available. 

Both  of  the.se  differences  are  .statistically  significant,  but  only  one  is  in  the  predicted  direction. 
Occupational  Injury  days  decreased  significantly  from  the  pre-  to  post-training  period.  Sick  days  lo.st,  on 
the  other  hand,  significantly  increa.sed  during  the  period.  Tlie  training  planners  and  managers  had 
predicted  that  improved  lost  time  due  to  injury  would  improve  more  readily  than  sick  days  lost  following 
the  training. 


Performance;  Personal 


Figure  7.11  Mean  .Scores,  I*re-  and  Post-Training,  for  Safety  of  Aircraft 
Performance  Measures 
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Measures  of 
Dependahililv,  Figure 
7.13  presents  before- 
and  after-training 
comparisons  for 
departures  within  5 
minutes,  within  15 
minutes,  within  6() 
minutes,  over  60 
minutes,  and  an 
overall  departure 
performance  measure 
which  includes  all  of 
the  above. 


Three  of  these  Figure  7.12  Mean  Percentage,  I’re-  and  Post-Training,  for  Personal  Safety 
differences  are  Performance  Measures 

statistically  significant, 

two  of  which  are  in  the  expected  direction.  Departures  within  5  minutes  and  the  overall  measure  improve 
significantly  from  pre-  to  po.st-training  periods  and  this  result  is  consistent  with  the  assessments  of  the 
trainers. 


““•p<  01.*-p<  05  'Januaiy  1992' 
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Percent  of  departures 
within  60  minutes 
significantly  decreases 
(the  reverse  direction 
to  that  predicted)  pre- 
to  post-training. 

Initially,  this  result 
appears  puzzling,  but, 
upon  refection,  it  is 
not  unexpected.  The 
two  performance 
indicators  (departures 
within  5  minutes,  and 
within  60  minutes), 
although  not 
completely 
independent  of  one 
another  in  their 

mea.surement,  can  vary  widely  in  practice.  To  illustrate  this,  consider  tlie  busy  summer  .season  when  all 
line  .statioas  .strive  for  on-time  performance.  Tlie  typical  station  (unable  to  substantially  increa.se  its 
manpower  or  overtime)  could  decide  to  sacrifice  the  schedule  of  aircraft  that  require  as  much  as  an  hour 
to  repair,  in  order  to  a.s.sign  the  line  maintenance  crews  to  those  aircraft  which  can  be  dispatched  "on- 
time."  This  tactic  causes  performance  on  the  5  minute  mark  to  improve  while  perlorinance  on  the  60 
minute  mark  declines. 
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Figure  7.13  Mean  Percentage,  Pre-  and  Post-Training,  for  Departure 
Performance 
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One  other  measure  of  depcndahiiity,  "expendable  parts  service  level,"  decreases  (an  unexpected  result) 
between  pre  and  post  periods  (pre=95%  <  post=93.3%;  Z=-1.83.  n=4,  p=.()7,).  The  other  measures  of 
dependability  --  "number  of  delays  due  to  late  from  maintenance"  (n=35),  "rotable  parts  service"  (n=8) 
and  "heavy  checks  on  time  to  initial  plan"  (n=4)  —  did  not  show  measurable  change  in  pre  and  post  levels 
of  performance. 

Measures  of 
Hfficiencv.  Figure 
7.14  shows  the  pre- 
post  comparisons  for 
the  percentage  of 
hours  applied  to 
production  and  for 
overtime  charged. 


Although,  neither 
measure  shows  change 
after  training,  the 
results  of  a  .series  of 
analy.ses  conducted  to 

assess  possible  Figure  7.14  Mean  Percentage.  I’re-  and  Post-Training,  for  Kfllcicncy 
relationships  between  Perforntance  Measures 

d  e  p  a  r  t  nt  e  n  t  and 
performance  measured 

revealed  that  only  overtime  charged  was  related  to  departmeiit.  These  results  show  significant  main  effects 
of  Pre-Post  change  (F=4.14.  P  <  .O.S)  and  department  (F  =  2.71,  p  <  .O.S),  indicating  all  departments, 
except  Quality  control,  reduced  overtime  after  uaining.  Figure  7.15  displays  the.se  changes  in  bar  chart 
form. 


7.6.4  Relationships 
Between  Post- 
training 
Attitudes  and 
Performance 


The  predictor  variables 
to  be  te.sted  include 
only  the  post-training 
results  of  the  four 
attitude  indices: 
"Sharing  Command 
Responsibility," 
"Communication  and 
Coordination," 
"Recognition  of 
Stressor  Hffects,"  and 
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"Willingness  to  Voice  Disagreement."  Spearman  Rank-order  Correlatiims  that  test  the  relationship  between 
these  indices  and  the  12  performance  measures  are  presented  in  Tables  7.2,  7.3,  and  7.4.  Becau.se  the 
pre.sent  data  are  limited  in  sample  si/.e  .stati.stical  significance  conventions  are  extended  to  show  not  only 
probability  levels  of  1%  and  .‘57r,  but  al.so  10%  and  \5%  to  aid  in  examining  patterns  of  expected  re.sults. 


A  r n  iTJDE  INDEX 
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"COMMUNICATION  AND 
COORDINATION" 

"RECOGNITION  OF 
STRESSOR  EFFECTS" 

"WILLINGNESS  TO  VOICE 
DISAGREEMENT" 

MEASURES  OF  SAFETY 

Numbar  of  Ground  Damage  Inddenta'  (no35} 

-.04 

-f.19 

-.03 

-^.30* 

%  OccupatfonaJ  Injury  Days  Lost'  (n-37) 

■••.28* 

■r.14 

-.09 

-.08 

%  Sick  Days  boat'  (r»s36) 

1  .2S* 

-.12 

-.02 

-.05 

**  p  <  .01;  ‘p  <  .05 

':'Thaaa  item*  score  lower  when  performance  is  positive,  thus  the  direction  of  the  statistics  are  mirrored  to  reflect 
positive  when  relationships  are  in  the  expected  direction. 

Table  7  2  Spearman-Rho  Correlations  Between  I’o.st-Training  CRM/TOQ  Results  and  Bo.st-Training 
Maintenance  Safety  Performance 

Tlie  matrix  of  results  reveals  that  post-training  attitudes  pre  lift  the  performance  of  the  work  units.  Tliese 
relationships  arc  e,sfK;cially  consistent  for  attitudes  regarding  the  willingness  of  units  to  share  command 
responsibility  and  to  voice  di.sagreement.  These  re.sults  are  di.scussed  below  in  more  detail. 

It’s  useful  to  note  that  the  total  number  of  po.sitive  and  significant  relationships  relative  to  their 
proportion  to  the  total  number  of  tests  presented  in  Tables  7.2,  7.3,  and  7.4  is  substantially  above  that 
expected  by  chance  alone.  Twenty  Spearman  "Rhos”  were  less  than  or  equal  to  a  probability  i)f .  I.*i.  from 
a  total  of  48  tests  conducted  (20/48  =  42%);  and  seven  of  them  were  less  than  .(),*>  (7/48  =  1,S%). 

Measures  of  Safety  Performance.  Po.st-training  responses  for  .some  of  the  altitude  indices  were  associated 
with  the  measures  of  Safely  performance  (.See  Table  7.2).  Specifically  "Willingness  to  Voice 
Disagreement"  was  related  to  the  number  of  ground-damage  incidents.  'Iliis  relationship  implies  that  the 
more  willing  managers  were  to  voice  disagreement  the  belter  that  their  units  performed  (e  g.,  had  lower 
incidents  of  ground  damage). 
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Similarly,  attitudes  about  "Sharing  Command  Responsibility"  were  pt)sitively  associated  with  percentage 
of  occupational  injury  days  lost.  That  is,  the  more  favorable  the  attitudes  were  about  "Sharing  Command 
Responsibility"  the  better  were  the  outcomes;  positive  attitudes  were  associated  with  fewer  days  lost  to 
occupational  injury.  Finally,  altitudes  on  this  same  index,  "Sharing  Command  Responsibility,"  were 
negatively  related  to  percentage  of  sick  days  lost.  This  relationship  is  not  in  the  expected  direction, 
suggesting  that  more  sick  days  arc  taken  among  units  who  are  more  willing  to  share  command 
respoasibility. 

Measures  of  Dependahilitv.  Departures  within  5  and  15  minutes  of  schedule  (the  two  mo.st  salient 
Dependability  measures)  and  60  minutes  display  significant  positive  relationships  with  most  or  all  of  the 
four  attitudes  (See  Table  7.3). 

Taken  together,  these  results  suggest  that  positive  attitudes  toward  the  concepts  learned  in  training  have 
a  facilitalive  effect  on  the  departure  [icrformance  of  units.  For  example,  those  units  favorable  towards  the 
sharing  command  responsibility  tend  to  have  better  departure  performance  measured  within  5,  15,  and  60 
minutes.  A  similar  pattern  of  positive  relationships  between  recognition  of  stressor  effects  and  these 
departure  performance  assessments  emerged  as  well.  I  Jne  managers  who  advtKate  the  importance  of 
recognizing  stress  are  more  likely  to  guide  their  mechanics  to  perform  well  on  the  percentage  of  goal 
achieved  for  departure  in  5,  15,  and  60  minutes. 

Warehouse  service  levels  show  few  positive  relationships  with  post-training  attitudes  (.sec  Table  7.3). 
Percentage  of  rotable  parts  available  is  moderately  correlated  (p<  .  10)  with  attitudes  on  the  index  "Sharing 
Command  Re.sponsibility."  Warehouse  service  levels  in  expendable  parts,  however,  show  a  negative 
relationship  (p<  .10)  to  "Willingness  to  Voice  Disagreement." 

Finally,  the  dependability  measure,  hcavy-chccks-on-time-to-plan,  .shows  two  perfect  positive  correlations 
with  "Sharing  Command  Responsibility"  and  "Recognition  of  Stressor  Hffects"  as  well  as  a  marginally 
significant  but  high  correlation  with  "Willingne.ss  to  Voice  Di.sagreement."  Although  the  number  of  units 
available  for  this  measurement  arc  unavoidably  low  (available  n=4)  the  consi.stent  pattern  of  positive 
relatiomships  suggest  that  positive  post-training  attitudes  are  related  to  achieving  the  standards  of  the  initial 
plan. 

Measures  of  FTficienev.  Table  7.4  .shows  that  attitudes  about  .sharing  command  responsibility  predict 
units’  performance  for  one  of  the  measures  of  efficiency,  "ratio  of  hours  paid  to  applied  to  pnxlucUon." 
This  relationship  suggests  that  the  more  the  unit  agrees  with  the  principle  of  sharing  command 
rc.sponsibility  the  better  will  be  the  ratio  of  efficiency.  Such  a  po.sitive  relation.ship,  however,  is  not 
evident  when  considering  the  percentage  of  ovcrtime-paid-to-total-wage.  In  fact,  there  is  a  nuxJe.st 
negative  relationship  between  this  efficiency  mca.sure  and  both  "Sharing  Command  Responsibility"  and 
"Willingness  to  Voice  Disagreement." 
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ATTITUDE  INDEX 

"SHARING  COMMAND"  "COMMUNICATION  AND  "RECOGNITION  OF  "WILLINGNESS  TO  VOICE 
RESPONSIBIUTY"  COORDINATION"  STRESSOR  EFFECTS"  DISAGREEMENT" 

MEASURES  OF  DEPENDABIUTY _ 

%  Departure  within  5  minutae  (n~26) 


+.32*  +,30t  +.27t  +.24* 


%  Haavy  checks  on  lima  to  initial  pian  (rt=4) 

+  1.0*  +.40  +1.0*  +.80t 

p  <  .01  *  p  <  .05  t  P  <  .10  ♦  P  <  -15 


Table  7.3  Spearman-Rho  Correlations  Between  Post-Training  CRM/TOQ  Re.sults  and  Post-Training 
Maintenance  Dependability  Performance 

7.7  QUAUTATIVE  DATA  OBTAINED  AFTER  THE  CRM  TRAINING 

Not  all  indicators  used  are  quantitative.  The  ca.se  .study  approach  is  also  beginning  to  provide  evidence 
for  the  program’s  success.  In  addition,  there  are  .several  open-ended  questions  included  in  the  immediate 
post-training  CRM/TOQ,  a.s  well  as  in  the  2-  and  6-month  follow-up  questionnaires.  Tlie.se  two  potential 
sources  of  evidence  will  be  de.scribed  and  illustrated  beh'w. 
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ATTITUDE  INDEX 


"SHARING  COMMAND" 
RESPONSIBIUTY" 

"COMMUNICATION  AND 
COORDINATION" 

"RECOGNITION  OF 
STRESSOR  EFFECTS" 

"WILLINGNESS  TO  VOICE 
DISAGREEMENT" 

MEASURES  OF  EFFICIENCY 

1  Rato  ot  Houra  paid  to  applied  to  production  (n>>6) 

II _ -zr _ 

+.23 

+  .20 

+.31 

1 

1  %  Overtime  pekt  to  total  wage'  (na>37) 

II _ _ 

-.09 

-.12 

-.204 

“  p  <  .01  *  p  <  .05  t  P  <  .10  4  P  <  -15 

These  items  score  lower  when  performance  is  positive,  thus  the  direction  of  the  statistics  are  mirrored  to  reflect 
positive  when  relationships  are  in  the  expected  direction. 

'Fable  7.4  Spearman- Rho  Correlations  Between  Post-Training  CRM/TOQ  Results  and  Post-Training 
Maintenance  Hlficiency  Performance 

7.7.1  The  Case  Study  Approach 

One  such  anecdote  involves  participant  reaction  to  the  stress-management  module  of  the  CRM  course,  and 
how  it  led  to  discovering  how  effective  the  course  could  be  in  heavy  maintenance  planning.  Meeting 
objectives  for  completion  times  (called  estimated  lime  for  return,  or  ”KTRs")  from  base  maintenance 
overhaul  are  always  raised  as  examples  of  "high  stre.ss"  parts  of  the  Job  by  managers  in  the  CRM  course. 
During  a  CRM  .se,ssion  in  November  1991,  .six  months  after  the  onset  of  the  CRM  training  in  Technical 
Operations,  Planning  and  Maintenance  Managers  from  one  maintenance  base  did  not  report  KTRs  as  a 
.source  of  stress.  With  a  little  encouragement  the.se  managers  revealed  that,  beginning  two  or  three  months 
before.  Maintenance  and  Planning  functions  meet  together  frequently  to  confirm  P'TRs,  or  to  change  them 
if  required.  'The  net  effect  is  an  improvement  not  only  in  the  performance  to  plan,  but  in  the  timeline.ss 
and  quality  of  the  aircraft  delivered  from  ba.se  maintenance.  'The  Director  of  Maintenance  for  that  base, 
it  turned  out,  had  attended  a  CRM  session  three  months  earlier  where  he  had  shown  an  increa.sed 
acceptance  for  the  concepts  --  having  ended  the  two  day  training  saying,  "Maybe  this  stuff  has  a  place  in 
management  after  all."  It  should  be  noted  that  quantitative  results,  pre.sented  earlier,  evidence  the 
relalion.ship  between  .several  attitude  indices  and  heavy  maintenance  performance  to  plan. 

7.7.2  Written  Comments  from  the  Post-training  and  Follow-up  CRM/TOQ 

"How  will  the  CRM  training  he  used  on  the  job?"  Immediately  following  the  training  this  question  was 
most  frequently  answered  "Better  communication"  (active  transferring  of  information),  "Better  listening" 
(a  passive  improvement  made  within  the  person).  "Being  more  aware  of  others"  (a  passive,  reactive 
behavior),  and  "Dealing  better  with  others"  (interactive,  problem-solving). 
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"How  did  you  use  the  CRM  Iraininu  on  the  joh?"  For  (he  saniple  ol  respondents  who  relumed  the  first 
2-monlh  and  6-nionth  Ibllow-up  questionnaires  (in  October,  1991)  the  largest  numbers,  saying  what  they 
had  used  from  the  CRM  training,  reported  actions  in  those  same  four  categories.  Tables  7.5  and  7.6  show 
lhe.se  compari.sons  for  the  immediate  Rost-training  survey  with  the  two-month  and  six-month  follow-up 
surveys,  re.spectively. 

Most  respondents  in  both  samples  emphasi/.ed  that  they  had  tried  to  li.sten  better,  and  to  "deal  better  with 
others"  (often  specifying  team  work,  decision  making  and  problem  solving  as  ways  of  doing  this).  The 
large.st  numbers  in  both  the  two-  and  .six-month  follow-up  samples  .stated  emphatically  that 
"Communicating  better,"  and  "Using  more  teamwork"  (the  latter  accounting  for  over  half  of  the  responses 
included  in  "dealing  belter  with  others")  was  what  they  intended  to  further  use  from  what  they  learned  in 
CRM  training. 

Tlie.se  open-ended  responses  lend  to  conlirm  that  improved  inlerper.sonal  behaviors  have  resulted  from  the 
positive  altitudes  which  followed  the  training.  Furthermore,  the  preferred  behaviors  lend  to  have  shifted 
from  tho.se  people  could  do  by  them.selves  (e.g.,  "be  a  better  li.stener"  and  being  more  aware  of  others), 
to  tho.se  behaviors  which  involve  others,  .such  as  "communicating  better,"  and  dealing  better  with  others. 


SURVEY  PERIOD 

RESPONSE 

POST-TRAINING 

WILL  USE 

TWO  MONTH 

WAS  USED 

TWO  MONTH 

WILL  USE 

Better  Communication 

20% 

15% 

17% 

Better  Listening 

16 

20 

3 

Be  More  Aware  of  Others 

11 

15 

11 

Deal  Better  with  Others 

10 

25 

25 

Use  in  Daily  Tasks 

6 

5 

3 

'Fable  7.5  Rercentage  of  Written-in  Responses  Indicating  How  Training  Will  be  U.sed  and  Was  Used 
on  the  Job  for  Ro.sl-lraining  and  Two  Month  Follow-up 


SURVEY  PERIOD 

RESPONSE 

POST-TRAINING 

WILL  USE 

SIX  MONTH 

WAS  USED 

SIX  MONTH 

WILL  USE 

Better  Communication 

20% 

20% 

26% 

Better  Listening 

16 

17 

6 

Be  More  Aware  of  Others 

1 1 

14 

6 

Deal  Better  with  Others 

10 

21 

23 

Use  in  Daily  Tasks 

6 

2 

2 

Table  7.6  Rercentage  of  Wrillen-in  Responses  Indicating  How  Training  Will  he  Used  and  Was  Used 
on  the  Job  tor  Rost -training  and  .Six  Month  Follow-up 
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"What  could  be  clone  to  improve  the  training?"  Most  answers  to  this  question,  immediately  after  training 
were  "do  nothing"  or  "it’s  fine  as  it  is."  Two  and  six  months  later  the  largest  single  category  of  answers 
was  "do  more  of  this"  or  "bring  us  back  for  follow-up  training."  Table  7.7  presents  these  results  for  both 
the  two  month  and  six  month  follow-up  surveys  compared  with  the  immediate  post  training  questionnaire. 
From  this  sample,  at  least,  participants  clearly  believe  they  would  benefit  from  further  CRM  type  training. 


SURVEY  PERIOD 

RESPONSE 

POST-TRAINING 

TWO  MONTH 
FOLLOW-UP 

SIX  MONTH 
FOLLOW-UP 

Needs  Nothing 

35% 

16% 

12% 

More  Training  &  Follow-up 

11 

27 

28 

More  Role  Playing 

7 

11 

4 

Add  Time  to  T raining 

7 

5 

4 

More  Case  Studies 

6 

7 

7 

Better  Mix  of  Participants 

6 

5 

7 

Table  7.7  Percentage  of  Written-in  Rc.spon,ses  Indicating  How  Training  Might  be  Improved  for  Post¬ 
training,  Two,  and  Six  Month  Follow-up 

7.8  CONCLUSIONS 

7.8.1  Strengths  of  the  CRM  Training  Program 

The  CRM  training  for  maintenance  managers  evaluated  here  has  been  successful,  after  as  short  a  time  as 

six  months,  in  improving  attitudes  which  appear  to  have  inllucnced  safe,  dependable  and  efficient 

performance.  Some  specific  findings  should  be  emphasized: 

1)  The  timing  and  content  of  the  program  has  been  well  received  by  particinanls.  Tlie  immediate 
evaluation  of  the  training  was  even  more  positive  than  had  been  the  ca.se  in  other  companies  using 
CRM  in  maintenance  as  well  as  flight  operations.  Figure  7.1  presented  this  dramatic  difference. 

2)  The  training  produces  a  .significant  improvement  in  most  attitudes  measured.  Changes  in  relevant 
attitudes  measured  immediately  before  and  after  training  reveal  strong  and  positive  changes  following 
training,  for  three  (T  the  four  indices  measured  (cf..  Figure  7.4).  Follow-up  re.sults  several  months 
after  training  tend  to  confirm  that  most  (if  not  all)  these  changes  are  stable  (Figures  7.9  and  7.10). 

3)  Performance  apoears  to  improve  due  to  CRM  training.  Changes  in  most  maintenance  performance 
mea.surcd  before  and  after  the  CRM  training  .sessions  examined  in  the  present  paper  show  significant 
changes  in  the  expected  direction  (e.g..  Figures  7.11-7.13). 
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4)  Specific  attitude  changes  seem  to  cause  specific  performance.  A  pattern  of  significant  relationships 
in  the  expected  direction  was  noted  between  the  post-training  attitudes  and  the  post-training 
performance.  Those  measures  predicted  (by  the  program’s  trainers  and  managers)  to  be  especially 
sensitive  to  the  effects  of  the  CRM  training  were  affected  (cf.,  Tables  7.2,  7.3).  In  particular,  positive 
attitudes  about  "Sharing  Command  Responsibility"  and  "Willingness  to  Voice  Disagreement"  both 
show  the  most  association  with  improved  performance. 

5)  More  active  change  is  occurring.  Finally,  the  program  is  reported  in  follow-up  surveys  to  be 
stimulating  changes  from  the  "easier"  more  passive  behaviors  (such  as  "better  listening"),  to  the  more 
interactive  ones  in  working  with  others,  such  as  holding  meetings  and  undertaking  joint  problem¬ 
solving.  Anecdotal  evidence  is  beginning  to  confirm  these  changes  and  their  impact  on  performance. 

Continuous  Improvement.  A  few  of  the  strengths,  such  as  the  final  one  noted  above,  also  provide 
guidance  where  the  training  design  for  this,  and  future  programs,  can  be  improved. 

The  results  presented  in  this  report  reflect  the  program  as  it  was  in  November,  1991.  The  training 
facilitators  and  administrators  of  the  program  receive  abundant  verbal  participant  feedback  at  the  end  of 
each  training  session.  They  have  also  received  the  earlier  reports  issued  through  this  research  project. 
A  watchword  of  those  managing  this  program  is  its  flexibility  and  adaptability  to  the  needs  (as  well  as 
constraints)  of  the  company,  as  well  as  to  increased  learning  about  this  new  kind  of  maintenance  training. 
Several  of  the  concerns  and  opportunities  noted  below  have  been  successfully  addressed  by  the  trainers 
since  November,  1991.  These  efforts  will  be  briefly  described. 

The  ambitious  syllabus  for  the  two-day  training  evaluated  here  (Appendix  A)  contains  many  important 
content  categories;  and  it  has  little  room  for  added  exercises  or  activities.  This  tight  program  has  created 
a  dilemma  for  the  trainers  from  its  onset.  For  them,  the  opportunities  for  program  improvement  usually 
require  classroom  time  to  exploit.  The  trainers  have  devised  an  innovative  solution  to  this  dilemma  in  at 
least  one  area  of  potential  improvement  described  below. 

7.8.2  Opportunities  for  Improvement  in  Maintenance  CRM  Training 

1)  Help  participants  plan  what  and  how  to  use  their  new-found  skills  back  at  work.  The  finding  that  a 
substantial  portion  of  participants  months  later  report  that  they  are  ready  and  willing  to  try  more  active 
behaviors  —  those  actually  involving  others  as  well  as  them.selves  -  suggests  that  the  training  program 
might  be  improved  to  help  the  "graduates"  to  design  or  develop  their  own  approach  to  implementation. 
Such  implementation  could  help  participants  actually  plan,  during  the  training,  how  and  what  they 
would  try  to  change  when  they  returned  to  work.  Part  of  this  awareness  could  include  research  results 
(such  as  those  contained  in  this  report)  for  participants  so  that  they  understand  that  listening  and 
stress-management  skills  are  useful  and  important,  but  that  the  assertiveness  and  team  leadership  skills 
could  impact  safety  even  more.  Another  avenue  to  impaci  active  improvement  would  be  to  expand 
the  course  module  on  assertiveness  training. 

2)  Focus  directly  on  assertiveness  skill  training.  The  research  evidence  presented  in  this  report  .shows 
that,  of  the  four  major  attitude  clusters  derived  from  the  CRM/TD  questionnaire,  "Willingness  to 
Voice  Disagreement"  forms  a  mixed  picture.  Although  assertiveness  is  mentioned  throughout  the 
training,  the  average  .scores  reveal  less  positive  attitudes  about  voicing  disagreement  (or  wanting  to 
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"avoid  conllict”)  l\)llowi»g  ihc  course  lhan  bcloro  it.  This  shiK  is  most  marked  lor  the  youngest  and 
the  oldest;  and  lor  llie  lowest  and  highesi  in  the  management  hierarchy.  Tlio.se  in  tlie  middle  ol  the 
age  range  and  liierarchy  tend  to  sliow  some  shill  toward  addressing  disagreements."  What  is  intended 
of  the  CRM  training  in  this  regard?  As  an  active  social  skill,  assertiveness  Idrms  a  theme  throughout 
the  two-day  program,  hut  it  hasn't  been  emphasized  at  the  expen.se  of  other  a.specl.s  ol  the  course. 
Perhaps  more  time  directly  with  assertiveness  skills  is  required.  Tliere  are  ample  materials  already 
developed  and  included  in  the  participants’  CRM  handbook,  which  contain  theory  and  skill  practice 
sections,  on  assertiveness,  but  limited  clas,sr(H)iu  lime  has  not  permitted  the  intensive  coverage  this 
material  deserves,  (ireater  use  ol  role-playing  exercises  could  al.so  help  participants’  learn  how  to  be 
a,s.sertive  and  participative,  and  to  give  them  confidence  that  they  can  skilllully  employ  these 
behaviors,  but  this  too  requires  time. 

Trainers  in  the  present  program  teel  very  strongly  about  the  need  lor  practical  application  ol  all  ol 
the  CRM  concepts  -  and  as.sertiveness  in  particuku".  They  are  al.so  sensitive  about  the  contrived 
nature  of  even  the  best  role-playing  ca.ses  as  .seen  by  the  very  pragmatic  audience  they  are  training. 
In  an  elTort  to  emphasize  "real  lite"  ratlier  than  contrived  situations,  the  trainers  have  replaced  one  ol 
the  two  role-plays  dealing  with  "supporting  and  conlronting  others"  (a  major  aspect  of  a.s.sertivene.ss 
skill)  with  intensive  small-group  and  whole-group  discussions  dealing  with  the  same  concepts,  but 
hK'u.sed  on  personal  illustrations  volunteered  Irom  the  participants.  It  is  too  soon  to  quantitatively 
mea.sure  the  elTects  ol  this  innovation,  but  trainers’  initial  reports  are  encouraging. 

The  results  ol  association  between  post-training  altitudes  and  unit  perldrmance  (i.e..  Table  7.2)  lend 
to  show  that  positive  feelings  about  assert ivene.ss  and  .sharing  authority  are  most  related  to  safely  and 
dependability.  Further  improving  and/or  expanding  the  training  design  could  well  leverage  this 
advantage  by  increasing  the  numbers  of  participants  with  a  positive  opinion  about  assertiveness. 

3)  Plan  and  publicize  recurrent  CRM  training.  Another  way  to  develop  active  leadership  and  Idllower 
skills,  while  recognizing  the  ambitious  .syllabus  in  a  two-day  training  course,  would  be  to  plan  and 
publicize  a  CRM  program  that  includes  a  follow-up  training  module  (or  even  succe.ssive  recurrent 
training).  Ciiven  that  many  CRM  graduates  .say  that  they  would  like  to  receive  additional  training  like 
this,  expansion  of  the  program  curriculum  could  provide  a  real  advantage.  For  instance,  the  CRM 
training  program  could  be  designed  to  first  awakci.  awarene.ss  of  CRM  and  develop  easily  attainable 
succe.s.ses  and  then  sub.seciuenlly  develop  and  exerci.se  team  and  assertiveness  skills. 
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Date 


Airlines 

Technical  Operations  Division 
CCC  Workshop  Survey 

BASELINE 

Please  answer  by  writing  beside  each  item  the  number  (hat  best  reflects  your  personal  opinion.  Choose  the  number  from 
the  scale  below, 

***  Scale  *** 

12  3  4  5 

Disagree  Disagree  Neutral  Agree  Agree 

Strongly  Slightly  Slightly  Strongly 


2  1  ■  Tech.  Ops.  team  members  should  avoid  disagreeing  with  others  because  conflicts  create  tension 
and  reduce  team  effectiveness. 

^  2.  It  is  important  to  avoid  negative  comments  about  the  procedures  and  techniques  of  other  team 
members. 

_5_3.  Casual,  social  conversation  on  the  job  during  periods  of  low  workload  can  improve  Tech.  Ops. 
team  coordination. 

4.  Good  communications  and  team  coordination  are  as  important  as  technical  profiency  for  aircraft 
safety  and  operational  effectiveness. 

_5_5.  We  should  be  aware  of  and  sensitive  to  the  personal  problems  of  other  Tech.  Ops.  team 
members. 

_5_6.  The  manager,  supervisor,  or  as.st.  supervisor  in  charge  should  take  hands-on  control  and  make 
all  decisions  in  emergency  and  non-standard  situations 

^  7.  The  manager,  supervisor,  or  addt,  supervi.sor  in  charge  should  verbalize  plans  for  procedures 
or  actions  and  should  be  sure  that  the  information  is  understood  and  acknowledged  by  the  other 
Tech.  Ops.  team  members. 

8.  Tech,  Ops.  team  members  should  not  question  the  decisions  or  actions  of  the  manager, 
supervi.sor,  or  a.sst.  supervisor  except  when  they  threaten  the  safety  of  the  operation. 

^  9.  Even  when  fatigued,  I  perform  effectively  during  critical  phases  of  work. 
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***  SCALE  *** 


1  2  3  4  5 

Disagree  Disagree  Neutral  Agree  Agree 

Strongly  Slightly  Slightly  Strongly 


S  10.  Managers,  supervisors  and  asst,  supervisors  should  encourage  questions  during  normal 
operations  and  in  special  situations. 

11.  There  are  no  circumstances  where  the  subordinate  should  assume  control  of  a  project. 

^  12.  A  debriefing  and  critique  of  procedures  and  decision  after  each  major  task  is  an  important  part 
of  developing  and  maintaining  effective  team  coordination. 

‘I  13.  Overall,  successful  Tech.  Ops.  management  is  primarily  a  function  of  the  manager's, 
supervisor’s,  or  asst,  supervisor’s  technical  proficiency. 

_iL_14.  Training  is  one  of  the  manager’s  most  important  responsibilities. 

15.  Because  individuals  function  less  effectively  under  high  su-ess,  good  team  coordination  is  more 
important  in  emergency  or  abnormal  situations. 

16.  The  start-of-shift  team  briefing  is  important  for  safety  and  for  effective  team  management. 

17.  Effective  team  coordination  requires  each  person  to  take  into  account  the  personalities  of  other 
team  members. 

18.  The  responsibilities  of  the  manager,  supervisor,  or  asst,  supervisor  include  coordination 
between  his  or  her  work  team  and  other  support  areas. 

^1  19.  A  truly  professional  manager,  supervisor  or  asst,  supervisor  can  leave  personal  problems 
behind. 

^  20.  My  decision-making  ability  is  as  good  in  abnormal  situations  as  in  routine  daily  operations. 
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***  SCALE 


1  2  3  4  5 

Disagree  Disagree  Neutral  Agree  Agree 

Strongly  Slightly  Slightly  Strongly 

In  the  following  questions,  "my  management  group"  refers  to  those  people  who  report  to  the  same 
manager  that  1  do. 

_ L_21 .  I  am  kept  informed  by  others  in  my  management  group  about  the  goals  and  objectives  of  this 

organization  (e.g.,  cost,  quality,  service,  etc.). 

--  22.  Work  goals  and  priorities  are  understood  and  agreed  to  by  members  of  my  management  group. 


In  the  following  items,  "my  work  group"  refers  to  those  people  who  report  to  me. 

_ 1_23.  Employees  in  my  work  group  receive  detailed  feedback  regarding  the  organization's 

performance. 

24.  If  employees  in  my  work  group  disagree  with  the  goal  and  priorities  that  have  been 
established,  they  feel  free  to  raise  their  concerns  with  supervision. 

_ L_25.  Employees  in  other  groups  within  Tech.  Ops.  plan  and  coordinate  their  activities  effectively 

together  with  people  in  my  work  group. 

26.  Employees  in  other  groups,  departments  and  divisions  through  the  company  act  as  if  they  share 
many  of  the  same  organizational  goals  that  we  do. 
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BACKGROUND  INFORMATION 


Year  of  Birth 
Total  Years  at 
Sex  (M  or  F) 


Current 

Department 

Maintenance 

Engineering 

Quality  Control 

Planning 

Logistics 

Shop 

Material  Services 

Job  Title: 

Years  in  present  position: _ 

_ 

Past  Experience/Training  (#  of  Years): 

Military 

Trade  School 

College 

v' 

Other  Airlines 
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Date 


AIRLINES 

TECHNICAL  OPERATION  DIVISION 
CCC  WORKSHOP  SURVEY 


"Two-month  Follow-up"  Questionnaire 


Please  enter  the  five  digit  Personal  Identification  Number  that  you  selected 
at  the  beginning  of  the  seminar. 

Identification  Code _  _  _  _  _  _ 


Now,  please  answer  by  writing  beside  each  item  the  number  that  best  reflects  your 
personal  attitude.  Choose  the  number  from  the  scale  below.  All  date  are  strictly 
confidential. 

•*  SCALE  ** 

1  2  3  4  5 

Disagree  Disagree  Neutral  Agree  Agree 

Strongly  Slightly  Slightly  Slnxi^ 

_ ^1,  Technical  Operations  team  members  should  avoid  disagreeing  with  others. 

^  2.  It  is  important  to  avoid  negative  comments  about  the  procedures  and  techniques 

of  other  team  members. 


_ 3.  Casual,  social  conversation  on  the  job  during  periods  of  low  worked  can  improve 

Technical  Operations  team  coordination. 

4.  Good  communications  and  team  coordination  are  as  important  as  technical 
proficiency  for  aircraft  safety  and  operational  effectiveness. 

^  5.  We  should  be  aware  of  and  sensitive  to  the  personal  problems  of  other  Technical 
Operations  team  members. 

_f/_6.  The  manager,  supervisor,  or  assistant  supervisor  in  charge  should  verbalize  plans 
for  procedures  or  actions  and  should  be  sure  that  the  information  is  understood 
and  acknowledged  by  the  other  Technical  Operations  team  members. 

^  7.  The  manager,  supervisor,  or  assistant  supervisor  in  charge  should  verbalize  plans 
for  procedures  or  actions  and  should  be  sure  that  the  information  is  understood 
and  acknowledged  by  the  other  Technical  Operations  team  members. 
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I 


1  2  3  4  5 

Disagree  Disagree  Neutral  Agree  Agree 

Strongly  Slightly  Slightly  Strongly 

.^_8.  Technical  Operations  team  members  should  not  question  the  decisions  or 
actions  of  the  manager,  supervisor,  or  assistant  supervisor  except  when  they 
threaten  the  safety  of  the  operation. 

^  9.  Even  when  fatigued,  1  perform  effectively  during  critical  phases  of  work. 


_^10. 
’  11. 

.5'  12 

.5'  13. 

£>  14. 
5  15. 

^  16. 

_ll7. 

^  18. 

/  19. 

_L-20. 


Managers,  supervisors,  and  assistant  supervisors  should  encourage  questions 
during  normal  operations  and  in  special  situations. 

There  are  no  circumstances  where  the  subordinate  should  assume  control  of  a 
project. 

A  debriefing  and  critique  of  procedures  and  decisions  after  each  major  task  is 
an  important  part  of  developing  and  maintaining  effective  team  coordination. 

Overall,  successful  Technical  Operations  management  is  primarily  a  function 
of  the  manager’s,  supervisor’s,  or  assistant  supervisor’s  technical  proficiency. 

Training  is  one  of  the  manager’s  most  important  responsibilities. 

Because  individuals  function  less  effectively  under  high  stress,  good  team 
coordination  is  more  important  in  emergency  or  abnormal  situations. 

The  start-of-shift  team  briefing  is  important  for  safety  and  for  effective  team 
management. 

Effective  team  coordination  requires  each  person  to  take  into  account  the 
personalities  of  other  team  members. 

The  responsibilities  of  the  manager,  supervisor,  or  assistant  supervi,sor  include 
coordination  between  his  or  her  work  team  and  other  support  areas. 

A  truly  professional  manager,  supervisor,  or  assistant  supervisor  can  leave 
personal  problems  behind. 

My  decision-making  is  as  good  in  abnomial  situations  as  in  routine  daily 
operations. 
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SCAI.E 


1 

2  3 

4 

5 

Disagree 

Disagree  Neutral 

Agree 

Agree 

.Strongly 

Slightly 

Slightly 

Strongly 

In  the  following  questions,  "my  management  group"  refers  to  those  people  who 

report  to  the  same  manager  that  I  do. 

l! 

_ L21.  I  am  kept  informed  by  others  in  my  management  group  about  the  goals  and 

objectives  of  this  organization  (e  g.,  cost,  quality,  service,  etc.). 

-  '  22.  Work  goals  and  priorities  arc  understood  and  agreed  to  by  members  of  my 

management  group. 

In  the  following  items,  "my  work  group"  refers  to  those  people  who  report  to  me. 

_ L_23.  Employees  in  my  work  group  receive  detailed  feedback  regarding  the 

organization’s  performance. 

_ _24.  If  employees  in  work  group  disagree  with  the  goals  and  priorities  that  have 

been  established,  they  feel  free  to  rai.se  their  concerns  with  supervision. 

_ j_25.  Employees  in  other  groups  within  Technical  Operations  plan  and  coordinate 

their  activities  effectively  together  with  people  in  my  work  group. 

_ 26.  Employees  in  other  groups,  departments  and  divisions  throughout  the  company 

act  as  if  they  share  many  of  the  same  organizational  goals  that  we  do. 

27.  Mow  useful  has  the  CCC  training  been  for  others?  (Circle  one) 

A  Waste  .Slightly  Somewhat  Very  Extremely 

of  Time  Useful  U.seful  Useful  Useful 

28.  Mow  much  has  the  CCC  training  changed  your  behavior  on  the  Job?  (Circle  one) 

No  Change  A  Slight  A  Moderate  A  Large 

Change  Change  Change 
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29.  What  changes  have  you  made  as  a  result  of  the  CCC  training? 


30.  How  will  you  further  use  the  CCC  training  in  the  coming  months? 


31.  Looking  back  on  it  now,  what  aspects  of  the  training  were  particularly  good? 


32.  What  do  you  think  could  be  done  to  improve  CCC  training? 
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Year  of  Birth 

Total  years  at 

5 

Sex  (M  or  F) 

A7 

CURRENT 

DEPARTMENT 

\' 

Line  Maintenance 

Base  Maintenance 

Quality  Control 

Planning 

Shop 

Material  Services 

Engineering 

Other 

Work  Location  -  City 


Job  Title: 

»r' 

Years  in  present  position:  2- 

Past  experience/training  (#  of  vears): 

Military  _ 

Trade  School  _ 

7 

College  _ 

Other  Airline  _ 

/ 

This  completes  the  questionnaire 
Thanks  for  your  help. 
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